2024-09-10T18:23:08.182 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:23:08.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:23:08.257 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7900238' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) mon_bind_msgr2: false sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 extra_packages: - cephadm selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGbgGTOnBRnUytMYlqIJVKTdhG/7Q4y8SERUHVwFY1nqTlSMICTvnoDF2XLcX0z9zIqhNqkZurTOntyLYiD2pMo= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfd8E7Fr9HIAanLo5MAjY13zyC6SgLsPle72PYlFHniSDNZJEidZLTpYXJLXQbNFmTo8BnPNxhhsTdkMNzVKaU= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:23:08.258 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:23:08.258 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:23:08.258 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:23:08.260 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:23:08.280 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:23:08.280 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:23:08.281 INFO:teuthology.packaging:ref: None 2024-09-10T18:23:08.281 INFO:teuthology.packaging:tag: None 2024-09-10T18:23:08.281 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:23:08.281 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:23:08.281 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:23:08.420 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:23:08.422 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:23:08.431 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:23:08.431 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:23:08.452 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:23:08.463 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:23:08.474 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:23:08.495 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi131.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:18:10.665283', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGbgGTOnBRnUytMYlqIJVKTdhG/7Q4y8SERUHVwFY1nqTlSMICTvnoDF2XLcX0z9zIqhNqkZurTOntyLYiD2pMo='} 2024-09-10T18:23:08.516 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:18:10.663863', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfd8E7Fr9HIAanLo5MAjY13zyC6SgLsPle72PYlFHniSDNZJEidZLTpYXJLXQbNFmTo8BnPNxhhsTdkMNzVKaU='} 2024-09-10T18:23:08.516 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:23:08.523 INFO:teuthology.task.internal:roles: ubuntu@smithi131.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:23:08.523 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:23:08.523 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:23:08.582 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdbc86e3ac0>, signals=[15]) 2024-09-10T18:23:08.583 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:23:08.591 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:23:08.591 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T18:23:08.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:23:08.667 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-09-10T18:23:08.668 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:23:08.741 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:23:08.748 DEBUG:teuthology.orchestra.run.smithi131:> uname -m 2024-09-10T18:23:08.767 INFO:teuthology.orchestra.run.smithi131.stdout:x86_64 2024-09-10T18:23:08.767 DEBUG:teuthology.orchestra.run.smithi131:> cat /etc/os-release 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:NAME="CentOS Stream" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION="9" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:ID="centos" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:VERSION_ID="9" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:ANSI_COLOR="0;31" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:23:08.825 INFO:teuthology.orchestra.run.smithi131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:23:08.826 INFO:teuthology.orchestra.run.smithi131.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:23:08.826 INFO:teuthology.orchestra.run.smithi131.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:23:08.826 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:23:08.826 INFO:teuthology.orchestra.run.smithi131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:23:08.826 INFO:teuthology.lock.ops:Updating smithi131.front.sepia.ceph.com on lock server 2024-09-10T18:23:08.845 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-09-10T18:23:08.862 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-09-10T18:23:08.863 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-09-10T18:23:08.920 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-09-10T18:23:08.920 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-09-10T18:23:08.920 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-09-10T18:23:08.920 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:23:08.920 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:23:08.921 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:23:08.921 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-09-10T18:23:08.945 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:23:08.955 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:23:08.961 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:23:08.962 DEBUG:teuthology.orchestra.run.smithi131:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:23:08.964 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:23:08.980 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:23:08.986 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:23:08.986 DEBUG:teuthology.orchestra.run.smithi131:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:23:09.022 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:23:09.140 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:23:09.194 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:23:09.209 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:23:09.209 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:23:09.209 DEBUG:teuthology.orchestra.run.smithi131:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:23:09.210 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:23:09.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:23:09.225 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-10T18:23:09.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:23:09.226 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-10T18:23:09.281 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:23:09.281 INFO:teuthology.task.kernel:Running kernel on smithi175: 5.14.0-503.el9.x86_64 2024-09-10T18:23:09.281 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum install -y kernel 2024-09-10T18:23:09.284 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:23:09.284 INFO:teuthology.task.kernel:Running kernel on smithi131: 5.14.0-503.el9.x86_64 2024-09-10T18:23:09.285 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum install -y kernel 2024-09-10T18:23:12.819 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T18:23:12.917 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T18:23:15.976 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-10T18:23:15.986 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-10T18:23:20.295 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T18:23:21.677 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.5 MB 00:01 2024-09-10T18:23:21.713 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 94 kB/s | 19 kB 00:00 2024-09-10T18:23:22.592 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T18:23:23.194 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-10T18:23:24.365 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-10T18:23:28.081 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:23:29.745 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T18:23:30.146 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:23:30.146 INFO:teuthology.orchestra.run.smithi175.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:23:30.237 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:23:30.246 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 79 M 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 126 M 2024-09-10T18:23:30.247 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:23:30.824 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2024-09-10T18:23:31.785 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:23:31.786 INFO:teuthology.orchestra.run.smithi131.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:23:31.877 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:23:31.893 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:23:31.894 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-09-10T18:23:31.894 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:23:31.894 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 79 M 2024-09-10T18:23:31.894 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 126 M 2024-09-10T18:23:31.894 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:23:32.055 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 162 kB/s | 23 kB 00:00 2024-09-10T18:23:32.355 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-10T18:23:32.747 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-10T18:23:33.097 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 27 MB/s | 29 MB 00:01 2024-09-10T18:23:33.098 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:23:33.098 INFO:teuthology.orchestra.run.smithi131.stdout:Total 65 MB/s | 79 MB 00:01 2024-09-10T18:23:33.190 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-09-10T18:23:33.787 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:23:34.161 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:23:34.161 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:23:35.132 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:23:35.132 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:23:37.323 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.4 MB/s | 29 MB 00:06 2024-09-10T18:23:37.598 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:23:38.318 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:23:38.406 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:23:38.656 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 4.1 MB/s | 35 MB 00:08 2024-09-10T18:23:38.657 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:23:38.657 INFO:teuthology.orchestra.run.smithi175.stdout:Total 9.4 MB/s | 79 MB 00:08 2024-09-10T18:23:39.336 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:23:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:23:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:23:40.637 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:23:40.638 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:23:41.210 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:23:41.341 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:23:43.076 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:23:43.715 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:23:43.797 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:23:46.152 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:23:46.433 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:23:46.588 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:23:46.717 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:23:51.256 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:23:51.537 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:23:51.576 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:23:56.414 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:29.700 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:29.740 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:30.025 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:30.025 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:24:30.025 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:24:30.025 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:24:30.852 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:24:30.853 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:24:31.211 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:24:31.300 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:31.301 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:24:31.301 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:24:31.301 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:24:31.301 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:24:31.301 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:24:31.301 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi131.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:24:31.302 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:24:31.361 INFO:teuthology.orchestra.run.smithi131.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:31.361 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:24:31.361 DEBUG:teuthology.orchestra.run.smithi131:> sudo rpm -qi grub2-tools 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Name : grub2-tools 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Epoch : 1 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Version : 2.06 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Release : 82.el9 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Architecture: x86_64 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Group : Unspecified 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:Size : 8274670 2024-09-10T18:24:31.401 INFO:teuthology.orchestra.run.smithi131.stdout:License : GPLv3+ 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Packager : builder@centos.org 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Vendor : CentOS 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Summary : Support tools for GRUB. 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:Description : 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:24:31.402 INFO:teuthology.orchestra.run.smithi131.stdout:hardware devices. 2024-09-10T18:24:31.403 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:24:31.403 INFO:teuthology.orchestra.run.smithi131.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:24:31.403 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:24:31.404 INFO:teuthology.task.kernel:Updating grub on smithi131 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:24:31.404 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:24:32.043 INFO:teuthology.orchestra.run.smithi131.stderr:Generating grub configuration file ... 2024-09-10T18:24:33.405 INFO:teuthology.orchestra.run.smithi131.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:24:33.439 INFO:teuthology.orchestra.run.smithi131.stderr:done 2024-09-10T18:24:33.441 DEBUG:teuthology.orchestra.run.smithi131:> mktemp 2024-09-10T18:24:33.457 INFO:teuthology.orchestra.run.smithi131.stdout:/tmp/tmp.m87l8RvkZk 2024-09-10T18:24:33.458 DEBUG:teuthology.orchestra.run.smithi131:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.m87l8RvkZk 2024-09-10T18:24:33.530 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0666 /tmp/tmp.m87l8RvkZk 2024-09-10T18:24:33.677 DEBUG:teuthology.orchestra.remote:smithi131:/tmp/tmp.m87l8RvkZk is 6KB 2024-09-10T18:24:33.725 DEBUG:teuthology.orchestra.run.smithi131:> rm -fr /tmp/tmp.m87l8RvkZk 2024-09-10T18:24:33.740 DEBUG:teuthology.orchestra.run.smithi131:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:24:33.803 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:24:33.803 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:24:33.803 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:24:33.803 INFO:teuthology.orchestra.run.smithi131.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:24:33.804 DEBUG:teuthology.orchestra.run.smithi131:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:24:33.949 DEBUG:teuthology.orchestra.run.smithi131:> sudo shutdown -r now 2024-09-10T18:24:34.740 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:34.780 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:24:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:24:35.042 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:24:35.891 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:24:36.236 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:24:36.317 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:36.317 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:24:36.317 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:24:36.317 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:24:36.317 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:24:36.317 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:24:36.317 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:24:36.318 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:24:36.376 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:36.376 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:24:36.377 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-09-10T18:24:36.418 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-09-10T18:24:36.418 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-09-10T18:24:36.418 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-09-10T18:24:36.418 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 82.el9 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8274670 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:24:36.419 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:24:36.420 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:24:36.421 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:24:36.421 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:24:36.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:24:37.112 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-09-10T18:24:38.511 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:24:38.538 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-09-10T18:24:38.541 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-10T18:24:38.557 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.595uyz5664 2024-09-10T18:24:38.557 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.595uyz5664 2024-09-10T18:24:38.626 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.595uyz5664 2024-09-10T18:24:38.767 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.595uyz5664 is 6KB 2024-09-10T18:24:38.817 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.595uyz5664 2024-09-10T18:24:38.832 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:24:38.894 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:24:38.895 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:24:38.895 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:24:38.895 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:24:38.896 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:24:39.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-09-10T18:25:03.957 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:25:03.957 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T18:25:03.958 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:25:09.057 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:25:09.057 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-10T18:25:09.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:25:43.741 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-10T18:25:52.751 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-10T18:25:52.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:26:03.969 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T18:26:11.133 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-09-10T18:26:12.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T18:26:12.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:26:13.522 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-10T18:26:14.192 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T18:26:14.192 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:26:14.193 DEBUG:teuthology.orchestra.run.smithi131:> uname -r 2024-09-10T18:26:14.207 INFO:teuthology.orchestra.run.smithi131.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:26:14.207 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:26:14.207 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:26:14.208 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:26:15.208 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:26:15.208 DEBUG:teuthology.orchestra.run.smithi131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:26:15.271 INFO:teuthology.orchestra.run.smithi131.stdout:ttyS1 2024-09-10T18:26:15.296 DEBUG:teuthology.parallel:result is None 2024-09-10T18:26:23.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-10T18:26:23.135 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:26:23.479 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-10T18:26:23.829 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-10T18:26:23.829 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:26:23.829 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-10T18:26:23.845 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:26:23.845 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:26:23.845 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:26:23.845 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:26:24.846 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:26:24.847 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:26:24.932 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-09-10T18:26:24.963 DEBUG:teuthology.parallel:result is None 2024-09-10T18:26:24.963 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:26:24.972 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:26:24.972 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:26:24.975 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:26:24.991 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:26:24.997 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:26:25.006 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:26:25.006 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:26:25.034 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:26:25.096 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:26:25.102 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:26:25.103 DEBUG:teuthology.orchestra.run.smithi131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:26:25.105 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:26:25.135 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:26:25.141 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:26:25.146 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:26:25.152 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:26:25.154 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:26:25.161 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:26:25.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:26:25.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:26:25.221 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:26:25.230 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:26:25.230 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:26:25.264 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:26:25.282 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:26:25.365 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:26:25.417 DEBUG:teuthology.orchestra.run.smithi131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:26:25.500 DEBUG:teuthology.orchestra.run.smithi131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:26:25.540 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:26:25.540 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:26:25.605 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:26:25.644 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:26:25.699 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:26:25.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:26:25.827 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:26:25.828 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:26:25.895 DEBUG:teuthology.orchestra.run.smithi131:> sudo service rsyslog restart 2024-09-10T18:26:25.898 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-09-10T18:26:25.938 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:26:25.982 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:26:26.369 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:26:26.378 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:26:26.378 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:26:26.387 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:26:26.396 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:26:26.396 DEBUG:teuthology.orchestra.run.smithi131:> sudo service auditd rotate 2024-09-10T18:26:26.468 INFO:teuthology.orchestra.run.smithi131.stdout:Rotating logs: 2024-09-10T18:26:26.470 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-09-10T18:26:26.545 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-09-10T18:26:26.547 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:26:26.547 DEBUG:teuthology.orchestra.run.smithi131:> /usr/sbin/getenforce 2024-09-10T18:26:26.581 INFO:teuthology.orchestra.run.smithi131.stdout:Permissive 2024-09-10T18:26:26.582 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-09-10T18:26:26.601 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-09-10T18:26:26.601 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi131.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:26:26.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:26:26.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:26:26.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:26:26.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:26:26.674 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:26:26.675 DEBUG:teuthology.orchestra.run.smithi131:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:26:26.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:26:26.767 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:26:26.777 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:26:26.790 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:26:26.791 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi131.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:30:09.919 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi131.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-09-10T18:30:09.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T18:30:09.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:30:10.000 DEBUG:teuthology.orchestra.run.smithi131:> true 2024-09-10T18:30:10.081 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi131.front.sepia.ceph.com' 2024-09-10T18:30:10.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-10T18:30:10.082 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:30:10.158 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-10T18:30:10.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-10T18:30:10.246 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:30:10.258 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:30:10.258 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:30:10.259 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:30:10.262 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:30:10.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:30:10.291 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:30:10.307 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:30:10.316 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:30:10.331 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:30:10.332 INFO:teuthology.orchestra.run.smithi131.stderr:sudo: ntpd: command not found 2024-09-10T18:30:10.343 INFO:teuthology.orchestra.run.smithi131.stdout:506 Cannot talk to daemon 2024-09-10T18:30:10.357 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-09-10T18:30:10.358 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:30:10.368 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-09-10T18:30:10.372 INFO:teuthology.orchestra.run.smithi131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:30:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:30:10.397 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:30:10.431 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-10T18:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.435 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.452 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:30:10.456 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:30:10.456 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-10T18:30:10.456 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.457 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.457 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.457 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:30:10.458 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:30:10.467 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:30:10.467 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T18:30:10.468 DEBUG:teuthology.orchestra.run.smithi131:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:30:10.468 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi175.front.sepia.ceph.com 2024-09-10T18:30:10.468 DEBUG:teuthology.orchestra.run.smithi175:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:30:10.826 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout:Removing: 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout:Removing unused dependencies: 2024-09-10T18:30:10.828 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout:Remove 2 Packages 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout:Freed space: 5.5 M 2024-09-10T18:30:10.829 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:30:10.832 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:30:10.833 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:30:10.848 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout:Removing: 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout:Removing unused dependencies: 2024-09-10T18:30:10.853 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout:Remove 2 Packages 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout:Freed space: 5.5 M 2024-09-10T18:30:10.854 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:30:10.857 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:30:10.857 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:30:10.883 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:30:10.884 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:30:10.919 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:30:10.919 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:30:11.031 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:30:11.053 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:30:11.109 INFO:teuthology.orchestra.run.smithi131.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:30:11.149 INFO:teuthology.orchestra.run.smithi175.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:30:11.165 INFO:teuthology.orchestra.run.smithi131.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:30:11.209 INFO:teuthology.orchestra.run.smithi175.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:30:11.552 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:30:11.552 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:30:11.704 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:30:11.705 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:30:11.816 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:30:11.816 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:11.816 INFO:teuthology.orchestra.run.smithi131.stdout:Removed: 2024-09-10T18:30:11.816 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:30:11.816 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:11.816 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:30:11.949 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:30:11.949 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:11.949 INFO:teuthology.orchestra.run.smithi175.stdout:Removed: 2024-09-10T18:30:11.949 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:30:11.949 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:11.950 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:30:12.658 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:29:23 PM UTC. 2024-09-10T18:30:12.768 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:29:23 PM UTC. 2024-09-10T18:30:12.799 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:30:12.800 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:12.800 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:30:12.800 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:12.800 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:30:12.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:30:12.802 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:12.802 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:30:12.802 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:12.802 INFO:teuthology.orchestra.run.smithi131.stdout:Install 7 Packages 2024-09-10T18:30:12.802 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:12.821 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 5.2 M 2024-09-10T18:30:12.822 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 19 M 2024-09-10T18:30:12.822 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:30:12.905 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:30:12.905 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-10T18:30:12.906 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout:Install 7 Packages 2024-09-10T18:30:12.907 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:12.931 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 5.2 M 2024-09-10T18:30:12.931 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 19 M 2024-09-10T18:30:12.932 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:30:13.104 INFO:teuthology.orchestra.run.smithi131.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 381 kB/s | 44 kB 00:00 2024-09-10T18:30:13.129 INFO:teuthology.orchestra.run.smithi131.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 746 kB/s | 106 kB 00:00 2024-09-10T18:30:13.179 INFO:teuthology.orchestra.run.smithi131.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-10T18:30:13.184 INFO:teuthology.orchestra.run.smithi175.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 398 kB/s | 44 kB 00:00 2024-09-10T18:30:13.205 INFO:teuthology.orchestra.run.smithi131.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 712 kB/s | 72 kB 00:00 2024-09-10T18:30:13.218 INFO:teuthology.orchestra.run.smithi175.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 728 kB/s | 106 kB 00:00 2024-09-10T18:30:13.260 INFO:teuthology.orchestra.run.smithi175.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 950 kB/s | 72 kB 00:00 2024-09-10T18:30:13.285 INFO:teuthology.orchestra.run.smithi175.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.2 MB/s | 84 kB 00:00 2024-09-10T18:30:13.322 INFO:teuthology.orchestra.run.smithi131.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.6 MB/s | 906 kB 00:00 2024-09-10T18:30:13.427 INFO:teuthology.orchestra.run.smithi175.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.5 MB/s | 906 kB 00:00 2024-09-10T18:30:13.472 INFO:teuthology.orchestra.run.smithi131.stdout:(6/7): runc-1.1.13-4.el9.x86_64.rpm 12 MB/s | 3.2 MB 00:00 2024-09-10T18:30:13.494 INFO:teuthology.orchestra.run.smithi175.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.5 MB/s | 837 kB 00:00 2024-09-10T18:30:13.531 INFO:teuthology.orchestra.run.smithi131.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 2.3 MB/s | 837 kB 00:00 2024-09-10T18:30:13.531 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:30:13.531 INFO:teuthology.orchestra.run.smithi131.stdout:Total 7.3 MB/s | 5.2 MB 00:00 2024-09-10T18:30:13.553 INFO:teuthology.orchestra.run.smithi175.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 12 MB/s | 3.2 MB 00:00 2024-09-10T18:30:13.553 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:30:13.553 INFO:teuthology.orchestra.run.smithi175.stdout:Total 8.3 MB/s | 5.2 MB 00:00 2024-09-10T18:30:13.640 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:30:13.655 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:30:13.655 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:30:13.660 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:30:13.674 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:30:13.674 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:30:13.964 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:30:13.965 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:30:14.001 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:30:14.001 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:30:14.440 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:30:14.464 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:30:14.525 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-10T18:30:14.555 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-10T18:30:14.565 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-10T18:30:14.596 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-10T18:30:14.598 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-10T18:30:14.626 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-10T18:30:14.811 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-10T18:30:14.835 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-10T18:30:14.844 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:30:14.866 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:30:15.297 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:30:15.313 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:30:15.322 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:30:15.344 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:30:15.435 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:30:15.459 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:30:15.500 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:30:15.522 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:30:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:30:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-10T18:30:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-10T18:30:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-10T18:30:16.474 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-10T18:30:16.474 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-10T18:30:16.474 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-10T18:30:16.544 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:30:16.544 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-10T18:30:16.545 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-10T18:30:16.545 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-10T18:30:16.545 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-10T18:30:16.545 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-10T18:30:16.545 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:16.828 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:16.892 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:30:17.211 DEBUG:teuthology.parallel:result is None 2024-09-10T18:30:17.284 DEBUG:teuthology.parallel:result is None 2024-09-10T18:30:17.284 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:30:17.292 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:30:17.292 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}, 'extra_packages': ['cephadm']} 2024-09-10T18:30:17.293 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:30:17.293 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:30:17.298 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:30:17.298 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:30:17.299 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:30:17.299 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:30:17.301 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:30:17.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:30:17.452 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:30:17.453 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:30:17.457 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:30:17.457 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:30:17.566 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:30:17.567 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:30:17.567 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:30:17.575 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:30:17.575 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:30:17.575 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:30:17.593 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:30:17.594 DEBUG:teuthology.orchestra.run.smithi131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:30:17.600 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:30:17.600 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:30:17.661 DEBUG:teuthology.orchestra.run.smithi131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:30:17.665 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:30:17.740 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:30:17.748 DEBUG:teuthology.orchestra.run.smithi131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:30:17.772 INFO:teuthology.orchestra.run.smithi131.stdout:check_obsoletes = 1 2024-09-10T18:30:17.773 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum clean all 2024-09-10T18:30:17.804 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-09-10T18:30:17.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-09-10T18:30:18.003 INFO:teuthology.orchestra.run.smithi131.stdout:45 files removed 2024-09-10T18:30:18.030 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-radosgw 2024-09-10T18:30:18.041 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-09-10T18:30:18.068 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw 2024-09-10T18:30:18.580 INFO:teuthology.orchestra.run.smithi131.stdout:ceph packages for x86_64 293 kB/s | 77 kB 00:00 2024-09-10T18:30:18.626 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 283 kB/s | 77 kB 00:00 2024-09-10T18:30:18.872 INFO:teuthology.orchestra.run.smithi131.stdout:ceph noarch packages 43 kB/s | 11 kB 00:00 2024-09-10T18:30:18.902 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 46 kB/s | 11 kB 00:00 2024-09-10T18:30:19.148 INFO:teuthology.orchestra.run.smithi131.stdout:ceph source packages 7.5 kB/s | 1.8 kB 00:00 2024-09-10T18:30:19.177 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 7.6 kB/s | 1.8 kB 00:00 2024-09-10T18:30:20.039 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.2 MB 00:00 2024-09-10T18:30:20.885 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.2 MB 00:01 2024-09-10T18:30:22.853 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-10T18:30:25.873 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 20 MB 00:04 2024-09-10T18:30:29.112 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.7 MB/s | 6.5 MB 00:02 2024-09-10T18:30:30.912 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-10T18:30:31.324 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 6.5 MB 00:01 2024-09-10T18:30:31.700 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T18:30:33.750 INFO:teuthology.orchestra.run.smithi131.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 19 kB 00:01 2024-09-10T18:30:34.538 INFO:teuthology.orchestra.run.smithi131.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-10T18:30:37.098 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-10T18:30:39.515 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:30:39.517 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:30:39.518 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout:Install 22 Packages 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2024-09-10T18:30:39.519 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:39.520 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 58 M 2024-09-10T18:30:39.520 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:30:40.177 INFO:teuthology.orchestra.run.smithi131.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T18:30:40.890 INFO:teuthology.orchestra.run.smithi175.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 5.7 MB/s | 5.0 MB 00:00 2024-09-10T18:30:40.923 INFO:teuthology.orchestra.run.smithi175.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 759 kB/s | 25 kB 00:00 2024-09-10T18:30:41.065 INFO:teuthology.orchestra.run.smithi175.stdout:(3/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 14 MB/s | 14 MB 00:01 2024-09-10T18:30:41.107 INFO:teuthology.orchestra.run.smithi175.stdout:(4/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 3.4 MB/s | 637 kB 00:00 2024-09-10T18:30:41.133 INFO:teuthology.orchestra.run.smithi175.stdout:(5/24): libradosstriper1-17.2.7-1606.g5be57395. 6.7 MB/s | 458 kB 00:00 2024-09-10T18:30:41.183 INFO:teuthology.orchestra.run.smithi175.stdout:(6/24): python3-ceph-argparse-17.2.7-1606.g5be5 906 kB/s | 45 kB 00:00 2024-09-10T18:30:41.425 INFO:teuthology.orchestra.run.smithi175.stdout:(7/24): ceph-common-17.2.7-1606.g5be57395.el9.x 16 MB/s | 22 MB 00:01 2024-09-10T18:30:41.450 INFO:teuthology.orchestra.run.smithi175.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 450 kB/s | 120 kB 00:00 2024-09-10T18:30:41.475 INFO:teuthology.orchestra.run.smithi175.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.1 MB/s | 156 kB 00:00 2024-09-10T18:30:41.501 INFO:teuthology.orchestra.run.smithi175.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:30:41.526 INFO:teuthology.orchestra.run.smithi175.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.7 MB/s | 295 kB 00:00 2024-09-10T18:30:41.551 INFO:teuthology.orchestra.run.smithi175.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:30:41.626 INFO:teuthology.orchestra.run.smithi175.stdout:(13/24): librgw2-17.2.7-1606.g5be57395.el9.x86_ 9.5 MB/s | 4.9 MB 00:00 2024-09-10T18:30:41.710 INFO:teuthology.orchestra.run.smithi175.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 872 kB/s | 160 kB 00:00 2024-09-10T18:30:41.735 INFO:teuthology.orchestra.run.smithi175.stdout:(15/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 246 kB/s | 45 kB 00:00 2024-09-10T18:30:41.802 INFO:teuthology.orchestra.run.smithi175.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 634 kB/s | 42 kB 00:00 2024-09-10T18:30:41.869 INFO:teuthology.orchestra.run.smithi175.stdout:(17/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2024-09-10T18:30:41.903 INFO:teuthology.orchestra.run.smithi175.stdout:(18/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.5 MB/s | 292 kB 00:00 2024-09-10T18:30:41.945 INFO:teuthology.orchestra.run.smithi175.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 718 kB/s | 49 kB 00:00 2024-09-10T18:30:41.970 INFO:teuthology.orchestra.run.smithi175.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.8 MB/s | 309 kB 00:00 2024-09-10T18:30:41.995 INFO:teuthology.orchestra.run.smithi175.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 818 kB/s | 67 kB 00:00 2024-09-10T18:30:42.045 INFO:teuthology.orchestra.run.smithi175.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-10T18:30:42.204 INFO:teuthology.orchestra.run.smithi175.stdout:(23/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 14 MB/s | 2.9 MB 00:00 2024-09-10T18:30:42.263 INFO:teuthology.orchestra.run.smithi175.stdout:(24/24): librados2-17.2.7-1606.g5be57395.el9.x8 11 MB/s | 3.1 MB 00:00 2024-09-10T18:30:42.268 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:30:42.268 INFO:teuthology.orchestra.run.smithi175.stdout:Total 21 MB/s | 58 MB 00:02 2024-09-10T18:30:42.332 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:30:42.366 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:30:42.366 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:30:42.554 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrading: 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:30:42.556 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:30:42.557 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout:Install 22 Packages 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout:Upgrade 2 Packages 2024-09-10T18:30:42.558 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:42.559 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 58 M 2024-09-10T18:30:42.559 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:30:42.845 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:30:42.845 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:30:43.412 INFO:teuthology.orchestra.run.smithi131.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 7.3 MB/s | 5.0 MB 00:00 2024-09-10T18:30:43.478 INFO:teuthology.orchestra.run.smithi131.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 378 kB/s | 25 kB 00:00 2024-09-10T18:30:43.595 INFO:teuthology.orchestra.run.smithi131.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 5.3 MB/s | 637 kB 00:00 2024-09-10T18:30:43.746 INFO:teuthology.orchestra.run.smithi131.stdout:(4/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 14 MB/s | 14 MB 00:01 2024-09-10T18:30:43.771 INFO:teuthology.orchestra.run.smithi131.stdout:(5/24): libradosstriper1-17.2.7-1606.g5be57395. 2.6 MB/s | 458 kB 00:00 2024-09-10T18:30:43.838 INFO:teuthology.orchestra.run.smithi131.stdout:(6/24): python3-ceph-argparse-17.2.7-1606.g5be5 679 kB/s | 45 kB 00:00 2024-09-10T18:30:44.031 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:30:44.080 INFO:teuthology.orchestra.run.smithi131.stdout:(7/24): ceph-common-17.2.7-1606.g5be57395.el9.x 17 MB/s | 22 MB 00:01 2024-09-10T18:30:44.114 INFO:teuthology.orchestra.run.smithi131.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 436 kB/s | 120 kB 00:00 2024-09-10T18:30:44.139 INFO:teuthology.orchestra.run.smithi131.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 2.6 MB/s | 156 kB 00:00 2024-09-10T18:30:44.164 INFO:teuthology.orchestra.run.smithi131.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:30:44.190 INFO:teuthology.orchestra.run.smithi131.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.7 MB/s | 295 kB 00:00 2024-09-10T18:30:44.215 INFO:teuthology.orchestra.run.smithi131.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:30:44.290 INFO:teuthology.orchestra.run.smithi131.stdout:(13/24): librgw2-17.2.7-1606.g5be57395.el9.x86_ 9.0 MB/s | 4.9 MB 00:00 2024-09-10T18:30:44.383 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:30:44.490 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:30:44.582 INFO:teuthology.orchestra.run.smithi131.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 123 kB/s | 45 kB 00:00 2024-09-10T18:30:44.590 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:30:44.685 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:30:44.724 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:30:44.749 INFO:teuthology.orchestra.run.smithi131.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 287 kB/s | 160 kB 00:00 2024-09-10T18:30:44.812 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:30:44.866 INFO:teuthology.orchestra.run.smithi131.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 362 kB/s | 42 kB 00:00 2024-09-10T18:30:44.867 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:30:44.891 INFO:teuthology.orchestra.run.smithi131.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 946 kB/s | 292 kB 00:00 2024-09-10T18:30:44.904 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:30:44.942 INFO:teuthology.orchestra.run.smithi131.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.0 MB/s | 662 kB 00:00 2024-09-10T18:30:45.025 INFO:teuthology.orchestra.run.smithi131.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 364 kB/s | 49 kB 00:00 2024-09-10T18:30:45.050 INFO:teuthology.orchestra.run.smithi131.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 309 kB 00:00 2024-09-10T18:30:45.076 INFO:teuthology.orchestra.run.smithi131.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 511 kB/s | 67 kB 00:00 2024-09-10T18:30:45.134 INFO:teuthology.orchestra.run.smithi131.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-10T18:30:45.301 INFO:teuthology.orchestra.run.smithi131.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-10T18:30:45.351 INFO:teuthology.orchestra.run.smithi131.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 11 MB/s | 2.9 MB 00:00 2024-09-10T18:30:45.356 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:30:45.357 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:30:45.357 INFO:teuthology.orchestra.run.smithi131.stdout:Total 21 MB/s | 58 MB 00:02 2024-09-10T18:30:45.394 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:30:45.422 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:30:45.455 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:30:45.456 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:30:45.456 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:30:45.493 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:30:45.536 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:30:45.573 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:30:45.635 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:30:45.664 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:30:45.721 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:30:45.762 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:30:45.805 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:30:45.845 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:30:45.938 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:30:45.938 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:30:46.150 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:30:46.192 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:30:46.282 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:30:46.356 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:30:46.386 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:30:47.134 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:30:47.519 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:30:47.579 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:30:47.681 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:30:47.757 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:30:47.797 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:30:47.886 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:30:47.931 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:30:47.977 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:30:48.413 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:30:48.442 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:30:48.505 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:30:48.541 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:30:48.590 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:30:48.630 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:30:48.691 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:30:48.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:30:48.786 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:30:48.826 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:30:48.861 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:30:48.895 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:30:48.898 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:30:48.932 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:30:49.194 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:30:49.241 INFO:teuthology.orchestra.run.smithi131.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:30:49.330 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:30:49.404 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:30:49.435 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:30:49.474 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:30:49.527 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:30:49.623 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:30:49.623 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:30:49.623 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:30:49.624 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:30:49.661 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:30:51.818 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:30:51.852 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:30:52.389 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:30:52.422 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:30:52.519 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:30:52.519 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:30:52.519 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:30:52.520 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:30:52.557 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:30:59.988 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-09-10T18:30:59.989 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:01.319 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:31:01.388 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:31:01.389 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:01.389 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:31:01.389 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:31:01.389 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:31:01.389 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:01.390 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:31:01.496 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:31:01.496 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:31:02.307 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:31:02.307 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /sys 2024-09-10T18:31:02.307 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /proc 2024-09-10T18:31:02.308 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /mnt 2024-09-10T18:31:02.308 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /var/tmp 2024-09-10T18:31:02.308 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /home 2024-09-10T18:31:02.308 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /root 2024-09-10T18:31:02.308 INFO:teuthology.orchestra.run.smithi131.stdout:skipping the directory /tmp 2024-09-10T18:31:02.309 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:02.740 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:31:02.740 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:31:02.740 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:31:02.740 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:31:02.740 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:31:02.740 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:31:02.741 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:31:02.742 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:31:03.507 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:31:03.508 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:31:03.509 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:03.510 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:31:03.510 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:03.510 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:03.617 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:31:03.695 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:31:03.696 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:03.696 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:31:03.696 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:31:03.696 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:31:03.696 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:03.697 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:31:03.795 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:31:03.795 INFO:teuthology.orchestra.run.smithi131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:31:03.886 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-test 2024-09-10T18:31:04.458 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:04.597 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:04.598 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:04.598 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:04.598 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:04.598 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:04.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:31:04.598 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:04.599 INFO:teuthology.orchestra.run.smithi175.stdout:Install 4 Packages 2024-09-10T18:31:04.600 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:04.600 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 48 M 2024-09-10T18:31:04.600 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 193 M 2024-09-10T18:31:04.600 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:04.845 INFO:teuthology.orchestra.run.smithi175.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.4 MB/s | 304 kB 00:00 2024-09-10T18:31:04.945 INFO:teuthology.orchestra.run.smithi175.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 636 kB/s | 64 kB 00:00 2024-09-10T18:31:04.970 INFO:teuthology.orchestra.run.smithi175.stdout:(3/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 657 kB/s | 163 kB 00:00 2024-09-10T18:31:05.055 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:31:05.055 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:31:05.055 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:31:05.056 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:31:05.057 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:31:05.872 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:31:05.872 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:05.872 INFO:teuthology.orchestra.run.smithi131.stdout:Upgraded: 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:31:05.873 INFO:teuthology.orchestra.run.smithi131.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:05.874 INFO:teuthology.orchestra.run.smithi131.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:31:05.875 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:05.875 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:06.237 INFO:teuthology.orchestra.run.smithi175.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-10T18:31:06.239 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:06.239 INFO:teuthology.orchestra.run.smithi175.stdout:Total 30 MB/s | 48 MB 00:01 2024-09-10T18:31:06.256 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-test 2024-09-10T18:31:06.267 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:06.286 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:06.286 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:06.653 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:06.654 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:06.829 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:26 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:06.971 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:31:06.972 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:31:06.973 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:06.973 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:06.973 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:06.973 INFO:teuthology.orchestra.run.smithi131.stdout:Install 4 Packages 2024-09-10T18:31:06.973 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:06.973 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 48 M 2024-09-10T18:31:06.974 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 193 M 2024-09-10T18:31:06.974 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:07.179 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:07.273 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:31:07.282 INFO:teuthology.orchestra.run.smithi131.stdout:(1/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 778 kB/s | 163 kB 00:00 2024-09-10T18:31:07.307 INFO:teuthology.orchestra.run.smithi131.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.3 MB/s | 304 kB 00:00 2024-09-10T18:31:07.327 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:31:07.356 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:31:07.433 INFO:teuthology.orchestra.run.smithi131.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 424 kB/s | 64 kB 00:00 2024-09-10T18:31:08.625 INFO:teuthology.orchestra.run.smithi131.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 31 MB/s | 48 MB 00:01 2024-09-10T18:31:08.627 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:08.627 INFO:teuthology.orchestra.run.smithi131.stdout:Total 29 MB/s | 48 MB 00:01 2024-09-10T18:31:08.655 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:08.674 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:08.674 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:09.039 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:09.040 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:09.563 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:09.666 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:31:09.707 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:31:09.735 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:31:11.780 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:31:11.861 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:31:14.133 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:31:14.174 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:31:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:31:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:31:14.175 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:31:14.208 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:14.487 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:14.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph 2024-09-10T18:31:15.361 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:15.507 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:31:15.509 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:31:15.510 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:31:15.511 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:31:15.512 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout:Install 34 Packages 2024-09-10T18:31:15.513 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:15.514 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 27 M 2024-09-10T18:31:15.514 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 96 M 2024-09-10T18:31:15.514 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:15.761 INFO:teuthology.orchestra.run.smithi175.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 56 kB/s | 6.5 kB 00:00 2024-09-10T18:31:15.953 INFO:teuthology.orchestra.run.smithi175.stdout:(2/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 6.7 MB/s | 2.1 MB 00:00 2024-09-10T18:31:15.995 INFO:teuthology.orchestra.run.smithi175.stdout:(3/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.0 MB/s | 1.4 MB 00:00 2024-09-10T18:31:16.087 INFO:teuthology.orchestra.run.smithi175.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 2.6 MB/s | 240 kB 00:00 2024-09-10T18:31:16.154 INFO:teuthology.orchestra.run.smithi175.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-10T18:31:16.446 INFO:teuthology.orchestra.run.smithi175.stdout:(6/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 31 MB/s | 15 MB 00:00 2024-09-10T18:31:16.472 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:31:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:31:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:31:16.473 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:31:16.792 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:16.793 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:17.088 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph 2024-09-10T18:31:17.146 INFO:teuthology.orchestra.run.smithi175.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 155 kB/s | 106 kB 00:00 2024-09-10T18:31:17.188 INFO:teuthology.orchestra.run.smithi175.stdout:(8/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 230 kB/s | 253 kB 00:01 2024-09-10T18:31:17.660 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:37 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:17.798 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:31:17.800 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:31:17.801 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:31:17.802 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout:Install 34 Packages 2024-09-10T18:31:17.803 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:17.804 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 27 M 2024-09-10T18:31:17.804 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 96 M 2024-09-10T18:31:17.804 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:17.880 INFO:teuthology.orchestra.run.smithi175.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 184 kB/s | 135 kB 00:00 2024-09-10T18:31:17.905 INFO:teuthology.orchestra.run.smithi175.stdout:(10/34): python3-requests-2.25.1-8.el9.noarch.r 174 kB/s | 125 kB 00:00 2024-09-10T18:31:18.130 INFO:teuthology.orchestra.run.smithi175.stdout:(11/34): python3-mako-1.1.4-6.el9.noarch.rpm 766 kB/s | 172 kB 00:00 2024-09-10T18:31:18.164 INFO:teuthology.orchestra.run.smithi175.stdout:(12/34): python3-cryptography-36.0.1-4.el9.x86_ 625 kB/s | 1.2 MB 00:02 2024-09-10T18:31:18.189 INFO:teuthology.orchestra.run.smithi175.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 594 kB/s | 35 kB 00:00 2024-09-10T18:31:18.248 INFO:teuthology.orchestra.run.smithi175.stdout:(14/34): python3-pytz-2021.1-5.el9.noarch.rpm 889 kB/s | 51 kB 00:00 2024-09-10T18:31:18.306 INFO:teuthology.orchestra.run.smithi175.stdout:(15/34): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-09-10T18:31:18.348 INFO:teuthology.orchestra.run.smithi175.stdout:(16/34): python3-packaging-20.9-5.el9.noarch.rp 420 kB/s | 77 kB 00:00 2024-09-10T18:31:18.373 INFO:teuthology.orchestra.run.smithi175.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 650 kB/s | 43 kB 00:00 2024-09-10T18:31:18.424 INFO:teuthology.orchestra.run.smithi175.stdout:(18/34): python3-cherrypy-18.6.1-2.el9.noarch.r 8.3 MB/s | 358 kB 00:00 2024-09-10T18:31:18.449 INFO:teuthology.orchestra.run.smithi175.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 465 kB/s | 11 kB 00:00 2024-09-10T18:31:18.474 INFO:teuthology.orchestra.run.smithi175.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 712 kB/s | 18 kB 00:00 2024-09-10T18:31:18.500 INFO:teuthology.orchestra.run.smithi175.stdout:(21/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-10T18:31:18.525 INFO:teuthology.orchestra.run.smithi175.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-09-10T18:31:18.550 INFO:teuthology.orchestra.run.smithi175.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T18:31:18.555 INFO:teuthology.orchestra.run.smithi131.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-10T18:31:18.575 INFO:teuthology.orchestra.run.smithi175.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-10T18:31:18.600 INFO:teuthology.orchestra.run.smithi175.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 925 kB/s | 46 kB 00:00 2024-09-10T18:31:18.626 INFO:teuthology.orchestra.run.smithi175.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T18:31:18.651 INFO:teuthology.orchestra.run.smithi175.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 661 kB/s | 16 kB 00:00 2024-09-10T18:31:18.677 INFO:teuthology.orchestra.run.smithi175.stdout:(28/34): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-09-10T18:31:18.710 INFO:teuthology.orchestra.run.smithi175.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-09-10T18:31:18.727 INFO:teuthology.orchestra.run.smithi175.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-09-10T18:31:18.748 INFO:teuthology.orchestra.run.smithi131.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-09-10T18:31:18.752 INFO:teuthology.orchestra.run.smithi175.stdout:(31/34): python3-urllib3-1.26.5-6.el9.noarch.rp 246 kB/s | 215 kB 00:00 2024-09-10T18:31:18.786 INFO:teuthology.orchestra.run.smithi175.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T18:31:18.806 INFO:teuthology.orchestra.run.smithi131.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 5.6 MB/s | 2.1 MB 00:00 2024-09-10T18:31:18.811 INFO:teuthology.orchestra.run.smithi175.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-10T18:31:18.837 INFO:teuthology.orchestra.run.smithi175.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.9 MB/s | 427 kB 00:00 2024-09-10T18:31:18.842 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:18.842 INFO:teuthology.orchestra.run.smithi175.stdout:Total 8.2 MB/s | 27 MB 00:03 2024-09-10T18:31:18.873 INFO:teuthology.orchestra.run.smithi131.stdout:(4/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 12 MB/s | 3.9 MB 00:00 2024-09-10T18:31:18.898 INFO:teuthology.orchestra.run.smithi131.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 2.6 MB/s | 240 kB 00:00 2024-09-10T18:31:18.974 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:19.012 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:19.012 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:19.023 INFO:teuthology.orchestra.run.smithi131.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-10T18:31:19.124 INFO:teuthology.orchestra.run.smithi131.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-09-10T18:31:19.291 INFO:teuthology.orchestra.run.smithi131.stdout:(8/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 28 MB/s | 15 MB 00:00 2024-09-10T18:31:19.316 INFO:teuthology.orchestra.run.smithi131.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 705 kB/s | 135 kB 00:00 2024-09-10T18:31:19.349 INFO:teuthology.orchestra.run.smithi131.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.7 MB/s | 1.2 MB 00:00 2024-09-10T18:31:19.392 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:19.392 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:19.433 INFO:teuthology.orchestra.run.smithi131.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 880 kB/s | 125 kB 00:00 2024-09-10T18:31:19.458 INFO:teuthology.orchestra.run.smithi131.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.5 MB/s | 215 kB 00:00 2024-09-10T18:31:19.642 INFO:teuthology.orchestra.run.smithi131.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 166 kB/s | 35 kB 00:00 2024-09-10T18:31:19.709 INFO:teuthology.orchestra.run.smithi131.stdout:(14/34): python3-packaging-20.9-5.el9.noarch.rp 308 kB/s | 77 kB 00:00 2024-09-10T18:31:19.742 INFO:teuthology.orchestra.run.smithi131.stdout:(15/34): python3-mako-1.1.4-6.el9.noarch.rpm 438 kB/s | 172 kB 00:00 2024-09-10T18:31:19.759 INFO:teuthology.orchestra.run.smithi131.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-10T18:31:19.810 INFO:teuthology.orchestra.run.smithi131.stdout:(17/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 649 kB/s | 43 kB 00:00 2024-09-10T18:31:19.827 INFO:teuthology.orchestra.run.smithi131.stdout:(18/34): python3-toml-0.10.2-6.el9.noarch.rpm 354 kB/s | 42 kB 00:00 2024-09-10T18:31:19.860 INFO:teuthology.orchestra.run.smithi131.stdout:(19/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-10T18:31:19.877 INFO:teuthology.orchestra.run.smithi131.stdout:(20/34): python3-jaraco-classes-3.2.1-5.el9.noa 1.2 MB/s | 18 kB 00:00 2024-09-10T18:31:19.902 INFO:teuthology.orchestra.run.smithi131.stdout:(21/34): python3-cherrypy-18.6.1-2.el9.noarch.r 3.9 MB/s | 358 kB 00:00 2024-09-10T18:31:19.919 INFO:teuthology.orchestra.run.smithi131.stdout:(22/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 116 kB/s | 11 kB 00:00 2024-09-10T18:31:19.936 INFO:teuthology.orchestra.run.smithi131.stdout:(23/34): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-10T18:31:19.953 INFO:teuthology.orchestra.run.smithi131.stdout:(24/34): python3-jaraco-functools-3.5.0-2.el9.n 582 kB/s | 19 kB 00:00 2024-09-10T18:31:19.970 INFO:teuthology.orchestra.run.smithi131.stdout:(25/34): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-10T18:31:19.987 INFO:teuthology.orchestra.run.smithi131.stdout:(26/34): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-10T18:31:20.004 INFO:teuthology.orchestra.run.smithi131.stdout:(27/34): python3-portend-3.1.0-2.el9.noarch.rpm 997 kB/s | 16 kB 00:00 2024-09-10T18:31:20.021 INFO:teuthology.orchestra.run.smithi131.stdout:(28/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T18:31:20.038 INFO:teuthology.orchestra.run.smithi131.stdout:(29/34): python3-tempora-5.0.0-2.el9.noarch.rpm 2.1 MB/s | 36 kB 00:00 2024-09-10T18:31:20.072 INFO:teuthology.orchestra.run.smithi131.stdout:(30/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-10T18:31:20.097 INFO:teuthology.orchestra.run.smithi131.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-10T18:31:20.122 INFO:teuthology.orchestra.run.smithi131.stdout:(32/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-10T18:31:20.147 INFO:teuthology.orchestra.run.smithi131.stdout:(33/34): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-10T18:31:20.173 INFO:teuthology.orchestra.run.smithi131.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T18:31:20.177 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:20.177 INFO:teuthology.orchestra.run.smithi131.stdout:Total 11 MB/s | 27 MB 00:02 2024-09-10T18:31:20.307 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:20.344 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:20.344 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:20.628 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:20.699 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:31:20.723 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:20.723 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:20.738 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:31:20.779 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:31:20.871 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:31:20.949 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:31:20.990 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:31:21.025 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:31:21.055 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:31:21.090 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:31:21.131 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:31:21.171 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:31:21.207 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:31:21.241 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:31:21.293 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:31:21.332 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:31:21.411 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:31:21.549 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:31:21.629 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:31:21.682 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:31:21.721 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:31:21.770 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:31:21.825 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:31:21.955 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:22.011 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:31:22.022 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:31:22.054 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:31:22.067 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:31:22.088 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:31:22.136 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:31:22.178 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:31:22.224 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:31:22.264 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:31:22.275 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:31:22.306 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:31:22.342 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:31:22.380 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:31:22.398 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:31:22.414 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:31:22.448 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:31:22.488 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:31:22.523 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:31:22.553 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:31:22.558 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:31:22.584 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:31:22.605 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:31:22.610 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:31:22.610 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:22.611 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:31:22.611 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:31:22.611 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:31:22.611 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:22.642 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:31:22.727 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:31:22.872 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:31:22.954 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:31:23.008 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:31:23.047 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:31:23.095 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:31:23.150 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:31:23.337 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:31:23.384 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:31:23.453 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:31:23.541 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:31:23.577 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:31:23.705 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:31:23.854 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:31:23.886 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:31:23.911 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:31:23.911 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:23.911 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:31:23.911 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:31:23.911 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:31:23.912 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:23.927 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:31:23.956 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:31:23.957 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:23.957 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:31:23.957 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:31:23.957 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:31:23.957 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:24.345 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:31:24.367 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:31:24.367 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:24.367 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:31:24.367 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:31:24.368 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:31:24.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:24.579 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:31:24.604 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:31:24.604 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:24.604 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:31:24.604 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:31:24.604 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:31:24.604 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:24.677 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:31:25.246 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:31:25.275 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:31:25.275 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:25.275 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:31:25.276 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:31:25.276 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:31:25.276 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:25.655 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:31:25.679 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:31:25.679 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:25.679 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:31:25.680 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:31:25.680 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:31:25.680 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:25.881 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:31:25.904 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:31:25.905 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:25.905 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:31:25.905 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:31:25.905 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:31:25.905 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:25.969 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:31:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:31:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:31:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:31:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:31:26.340 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:31:26.341 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:31:26.342 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:31:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:31:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:31:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:31:26.343 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:31:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:31:26.344 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:31:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:31:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:27.378 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:27.378 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:31:27.379 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:31:27.380 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:31:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:31:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:31:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:31:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:31:27.381 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:27.381 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:27.758 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:31:27.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:31:27.758 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:31:27.759 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:31:27.760 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:31:27.761 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:31:27.761 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:31:27.761 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:31:27.762 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:31:27.762 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:31:27.762 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:31:27.762 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:31:27.762 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:31:27.766 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-base 2024-09-10T18:31:28.339 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:28.431 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:31:28.473 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:28.474 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:31:28.474 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:28.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install cephadm 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:28.551 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:31:28.552 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:31:28.553 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:31:28.553 INFO:teuthology.orchestra.run.smithi131.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:31:28.553 INFO:teuthology.orchestra.run.smithi131.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:31:28.553 INFO:teuthology.orchestra.run.smithi131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:31:28.553 INFO:teuthology.orchestra.run.smithi131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:31:28.554 INFO:teuthology.orchestra.run.smithi131.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:31:28.554 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:31:28.554 INFO:teuthology.orchestra.run.smithi131.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:31:28.554 INFO:teuthology.orchestra.run.smithi131.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:31:28.554 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:31:28.554 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:31:28.555 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:28.556 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:28.907 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-base 2024-09-10T18:31:29.101 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:29.235 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:29.235 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 83 k 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 375 k 2024-09-10T18:31:29.236 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:29.437 INFO:teuthology.orchestra.run.smithi175.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 413 kB/s | 83 kB 00:00 2024-09-10T18:31:29.437 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:29.438 INFO:teuthology.orchestra.run.smithi175.stdout:Total 409 kB/s | 83 kB 00:00 2024-09-10T18:31:29.438 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:29.443 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:29.443 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:29.445 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:29.445 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:29.476 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:29.523 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:29.566 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:31:29.607 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:29.632 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:31:29.632 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:29.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install cephadm 2024-09-10T18:31:29.962 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:30.049 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:30.228 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:30.360 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:30.360 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 83 k 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 375 k 2024-09-10T18:31:30.361 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:30.438 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 427 kB/s | 83 kB 00:00 2024-09-10T18:31:30.555 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:Total 424 kB/s | 83 kB 00:00 2024-09-10T18:31:30.556 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:30.561 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:30.563 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:30.629 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:30.698 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:30.698 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:30.698 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:30.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:31:31.097 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:31.168 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:31.503 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:31.520 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:31.635 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:31.636 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:31.636 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-10T18:31:31.636 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:31.636 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 137 k 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 379 k 2024-09-10T18:31:31.637 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:31.763 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:31:31.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:31.763 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:31.763 INFO:teuthology.orchestra.run.smithi131.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:31.763 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:31.763 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:31.826 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 731 kB/s | 137 kB 00:00 2024-09-10T18:31:31.826 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:31.826 INFO:teuthology.orchestra.run.smithi175.stdout:Total 725 kB/s | 137 kB 00:00 2024-09-10T18:31:31.826 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:31.834 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:31.834 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:31.896 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:31.897 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:31.959 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:31:32.060 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:32.135 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:31:32.159 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:31:32.160 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:32.160 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:31:32.160 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:32.527 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:32.665 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 137 k 2024-09-10T18:31:32.666 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 379 k 2024-09-10T18:31:32.667 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:32.879 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 649 kB/s | 137 kB 00:00 2024-09-10T18:31:32.879 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:32.879 INFO:teuthology.orchestra.run.smithi131.stdout:Total 644 kB/s | 137 kB 00:00 2024-09-10T18:31:32.880 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:32.887 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:32.887 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:32.948 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:32.948 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:33.103 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:31:33.123 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:33.123 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:33.124 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:33.124 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:33.154 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:31:33.179 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:31:33.179 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:31:33.179 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:31:33.179 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:33.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr 2024-09-10T18:31:33.923 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:34.016 INFO:teuthology.orchestra.run.smithi175.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:31:34.059 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:34.060 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:31:34.060 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:34.087 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:31:34.087 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:34.087 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:31:34.087 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:34.087 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:34.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:31:34.313 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr 2024-09-10T18:31:34.685 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:00:57 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:34.816 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:31:34.817 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout:Install 7 Packages 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:34.818 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 3.7 M 2024-09-10T18:31:34.819 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 80 M 2024-09-10T18:31:34.819 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:34.897 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:34.987 INFO:teuthology.orchestra.run.smithi131.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:31:35.029 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:35.029 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:31:35.030 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:35.096 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:31:35.186 INFO:teuthology.orchestra.run.smithi175.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 84 kB/s | 15 kB 00:00 2024-09-10T18:31:35.220 INFO:teuthology.orchestra.run.smithi175.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1606.g5be 117 kB/s | 24 kB 00:00 2024-09-10T18:31:35.262 INFO:teuthology.orchestra.run.smithi175.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 120 kB/s | 9.0 kB 00:00 2024-09-10T18:31:35.287 INFO:teuthology.orchestra.run.smithi175.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:31:35.312 INFO:teuthology.orchestra.run.smithi175.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T18:31:35.338 INFO:teuthology.orchestra.run.smithi175.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:31:35.413 INFO:teuthology.orchestra.run.smithi175.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.4 MB/s | 3.4 MB 00:00 2024-09-10T18:31:35.416 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:35.417 INFO:teuthology.orchestra.run.smithi175.stdout:Total 6.1 MB/s | 3.7 MB 00:00 2024-09-10T18:31:35.417 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:35.425 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:35.425 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:35.506 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:35.506 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:35.674 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:35.806 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:31:35.807 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout:Installing weak dependencies: 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout:Install 7 Packages 2024-09-10T18:31:35.808 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:35.809 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 3.7 M 2024-09-10T18:31:35.809 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 80 M 2024-09-10T18:31:35.809 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:35.813 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:35.864 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:31:35.907 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:31:35.954 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:31:35.983 INFO:teuthology.orchestra.run.smithi131.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 103 kB/s | 15 kB 00:00 2024-09-10T18:31:35.993 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:31:36.008 INFO:teuthology.orchestra.run.smithi131.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1606.g5be 147 kB/s | 24 kB 00:00 2024-09-10T18:31:36.022 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:31:36.050 INFO:teuthology.orchestra.run.smithi131.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T18:31:36.075 INFO:teuthology.orchestra.run.smithi131.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:31:36.101 INFO:teuthology.orchestra.run.smithi131.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T18:31:36.126 INFO:teuthology.orchestra.run.smithi131.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:31:36.234 INFO:teuthology.orchestra.run.smithi131.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.5 MB/s | 3.4 MB 00:00 2024-09-10T18:31:36.238 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:36.238 INFO:teuthology.orchestra.run.smithi131.stdout:Total 8.5 MB/s | 3.7 MB 00:00 2024-09-10T18:31:36.238 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:36.246 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:36.246 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:36.337 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:36.337 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:36.642 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:36.719 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:31:36.753 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:31:36.808 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:31:36.848 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:31:36.877 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:31:37.027 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:31:37.067 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:31:37.861 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:31:37.897 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:31:38.090 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:31:38.090 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:31:38.091 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:31:38.091 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:31:38.091 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:31:38.091 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:31:38.091 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:31:38.364 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:31:38.364 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:38.365 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:38.607 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:31:38.950 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:31:39.176 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:02 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:39.223 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:39.224 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:39.310 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:39.311 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-10T18:31:39.311 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:39.311 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:31:39.312 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout:Install 12 Packages 2024-09-10T18:31:39.313 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:39.314 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 38 M 2024-09-10T18:31:39.314 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 212 M 2024-09-10T18:31:39.314 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:39.446 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:31:39.929 INFO:teuthology.orchestra.run.smithi175.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 888 kB/s | 191 kB 00:00 2024-09-10T18:31:39.996 INFO:teuthology.orchestra.run.smithi175.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 491 kB/s | 33 kB 00:00 2024-09-10T18:31:40.005 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:40.046 INFO:teuthology.orchestra.run.smithi175.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 801 kB 00:00 2024-09-10T18:31:40.138 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:40.139 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-10T18:31:40.139 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:40.139 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-10T18:31:40.139 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:40.139 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:31:40.139 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:31:40.140 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout:======================================================================================= 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout:Install 12 Packages 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 38 M 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 212 M 2024-09-10T18:31:40.141 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:40.171 INFO:teuthology.orchestra.run.smithi175.stdout:(4/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-10T18:31:40.205 INFO:teuthology.orchestra.run.smithi175.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 115 kB/s | 18 kB 00:00 2024-09-10T18:31:40.264 INFO:teuthology.orchestra.run.smithi175.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 419 kB/s | 38 kB 00:00 2024-09-10T18:31:40.356 INFO:teuthology.orchestra.run.smithi175.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-10T18:31:40.418 INFO:teuthology.orchestra.run.smithi131.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-10T18:31:40.476 INFO:teuthology.orchestra.run.smithi131.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-10T18:31:40.664 INFO:teuthology.orchestra.run.smithi175.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 4.5 MB/s | 3.0 MB 00:00 2024-09-10T18:31:40.748 INFO:teuthology.orchestra.run.smithi175.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-10T18:31:40.760 INFO:teuthology.orchestra.run.smithi131.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-10T18:31:40.844 INFO:teuthology.orchestra.run.smithi131.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 77 kB/s | 33 kB 00:00 2024-09-10T18:31:40.869 INFO:teuthology.orchestra.run.smithi131.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 168 kB/s | 18 kB 00:00 2024-09-10T18:31:40.936 INFO:teuthology.orchestra.run.smithi131.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 418 kB/s | 38 kB 00:00 2024-09-10T18:31:41.044 INFO:teuthology.orchestra.run.smithi131.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.3 MB/s | 3.0 MB 00:00 2024-09-10T18:31:41.070 INFO:teuthology.orchestra.run.smithi131.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-10T18:31:41.153 INFO:teuthology.orchestra.run.smithi131.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 250 kB 00:00 2024-09-10T18:31:41.271 INFO:teuthology.orchestra.run.smithi131.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 12 MB/s | 4.9 MB 00:00 2024-09-10T18:31:41.337 INFO:teuthology.orchestra.run.smithi131.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-10T18:31:41.453 INFO:teuthology.orchestra.run.smithi175.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.0 MB/s | 4.9 MB 00:01 2024-09-10T18:31:41.625 INFO:teuthology.orchestra.run.smithi175.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.0 MB/s | 5.1 MB 00:01 2024-09-10T18:31:41.649 INFO:teuthology.orchestra.run.smithi131.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-10T18:31:41.649 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:41.649 INFO:teuthology.orchestra.run.smithi131.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-10T18:31:42.005 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:42.031 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:42.032 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:42.519 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:42.519 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:43.283 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:43.354 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:31:43.397 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:31:43.493 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:31:43.537 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:31:43.720 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:31:43.761 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:31:43.838 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:31:44.477 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:31:44.590 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:31:45.048 INFO:teuthology.orchestra.run.smithi175.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.8 MB/s | 16 MB 00:04 2024-09-10T18:31:45.049 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:45.049 INFO:teuthology.orchestra.run.smithi175.stdout:Total 6.7 MB/s | 38 MB 00:05 2024-09-10T18:31:45.409 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:45.436 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:45.436 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:45.837 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:31:45.951 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:45.951 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:46.706 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:46.782 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:31:46.827 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:31:46.869 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:31:46.894 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:31:46.928 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:31:46.964 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:31:47.149 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:31:47.179 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:31:47.269 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:31:47.921 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:31:48.034 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:31:49.271 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:31:49.560 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:31:49.560 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:31:49.560 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:31:49.560 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:31:49.560 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:31:49.560 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:31:49.561 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:31:49.561 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:31:49.561 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:31:49.561 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:31:49.561 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:31:49.561 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:31:49.938 INFO:teuthology.orchestra.run.smithi131.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:31:49.939 INFO:teuthology.orchestra.run.smithi131.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:31:49.939 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:31:49.939 INFO:teuthology.orchestra.run.smithi131.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:31:49.939 INFO:teuthology.orchestra.run.smithi131.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:31:49.939 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:49.939 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:50.310 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:31:50.336 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:31:50.386 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:31:50.954 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:10 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:51.089 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:51.090 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:51.090 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:31:51.091 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout:Install 13 Packages 2024-09-10T18:31:51.092 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:51.093 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.2 M 2024-09-10T18:31:51.093 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 26 M 2024-09-10T18:31:51.093 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:51.469 INFO:teuthology.orchestra.run.smithi131.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 234 kB/s | 19 kB 00:00 2024-09-10T18:31:51.494 INFO:teuthology.orchestra.run.smithi131.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 239 kB/s | 26 kB 00:00 2024-09-10T18:31:51.561 INFO:teuthology.orchestra.run.smithi131.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-09-10T18:31:51.587 INFO:teuthology.orchestra.run.smithi131.stdout:(4/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 238 kB/s | 47 kB 00:00 2024-09-10T18:31:51.612 INFO:teuthology.orchestra.run.smithi131.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-10T18:31:51.637 INFO:teuthology.orchestra.run.smithi131.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-10T18:31:51.662 INFO:teuthology.orchestra.run.smithi131.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-10T18:31:51.705 INFO:teuthology.orchestra.run.smithi131.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-10T18:31:51.721 INFO:teuthology.orchestra.run.smithi131.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-10T18:31:51.738 INFO:teuthology.orchestra.run.smithi131.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-10T18:31:51.764 INFO:teuthology.orchestra.run.smithi131.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 2.5 MB/s | 220 kB 00:00 2024-09-10T18:31:51.789 INFO:teuthology.orchestra.run.smithi131.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-10T18:31:51.831 INFO:teuthology.orchestra.run.smithi131.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 8.2 MB/s | 1.0 MB 00:00 2024-09-10T18:31:51.833 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:51.834 INFO:teuthology.orchestra.run.smithi131.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-10T18:31:51.896 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:51.911 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:51.911 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:51.999 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:52.000 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:52.500 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:52.670 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:31:52.731 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:31:52.787 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:31:52.825 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:31:52.866 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:31:52.961 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:31:52.961 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:31:52.961 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:31:52.962 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:31:52.968 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:31:53.078 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:31:53.127 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:31:53.386 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:31:53.386 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:53.386 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:53.386 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:53.386 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:31:53.386 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:53.387 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:53.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:31:53.836 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:31:53.882 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:31:53.917 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:31:53.961 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:31:53.986 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:31:54.316 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:17 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:54.458 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:54.459 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:31:54.460 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:54.461 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:Install 13 Packages 2024-09-10T18:31:54.462 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.2 M 2024-09-10T18:31:54.462 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 26 M 2024-09-10T18:31:54.463 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:54.753 INFO:teuthology.orchestra.run.smithi175.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 177 kB/s | 19 kB 00:00 2024-09-10T18:31:54.777 INFO:teuthology.orchestra.run.smithi175.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 192 kB/s | 26 kB 00:00 2024-09-10T18:31:54.803 INFO:teuthology.orchestra.run.smithi175.stdout:(3/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 299 kB/s | 47 kB 00:00 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:31:54.814 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:31:54.815 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:31:54.815 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:31:54.903 INFO:teuthology.orchestra.run.smithi175.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-10T18:31:54.928 INFO:teuthology.orchestra.run.smithi175.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-10T18:31:54.962 INFO:teuthology.orchestra.run.smithi175.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 917 kB/s | 54 kB 00:00 2024-09-10T18:31:54.995 INFO:teuthology.orchestra.run.smithi175.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 481 kB/s | 32 kB 00:00 2024-09-10T18:31:55.021 INFO:teuthology.orchestra.run.smithi175.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 248 kB/s | 14 kB 00:00 2024-09-10T18:31:55.046 INFO:teuthology.orchestra.run.smithi175.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-10T18:31:55.071 INFO:teuthology.orchestra.run.smithi175.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.9 MB/s | 220 kB 00:00 2024-09-10T18:31:55.105 INFO:teuthology.orchestra.run.smithi175.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 3.3 MB/s | 90 kB 00:00 2024-09-10T18:31:55.130 INFO:teuthology.orchestra.run.smithi175.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 705 kB/s | 59 kB 00:00 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:31:55.161 INFO:teuthology.orchestra.run.smithi131.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:55.162 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:31:55.172 INFO:teuthology.orchestra.run.smithi175.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-10T18:31:55.175 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:55.175 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.1 MB/s | 2.2 MB 00:00 2024-09-10T18:31:55.241 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:31:55.256 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:31:55.256 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:31:55.346 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:31:55.347 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:31:55.482 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:31:55.852 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:31:56.034 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:31:56.051 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:16 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:31:56.089 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:31:56.145 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:31:56.184 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:31:56.185 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout:Install 6 Packages 2024-09-10T18:31:56.186 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:31:56.187 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 7.0 M 2024-09-10T18:31:56.187 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 33 M 2024-09-10T18:31:56.187 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:31:56.212 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:31:56.257 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:31:56.377 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:31:56.476 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:31:56.526 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:31:56.551 INFO:teuthology.orchestra.run.smithi131.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 578 kB/s | 124 kB 00:00 2024-09-10T18:31:56.577 INFO:teuthology.orchestra.run.smithi131.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-09-10T18:31:56.627 INFO:teuthology.orchestra.run.smithi131.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-10T18:31:56.652 INFO:teuthology.orchestra.run.smithi131.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 765 kB/s | 58 kB 00:00 2024-09-10T18:31:56.678 INFO:teuthology.orchestra.run.smithi131.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-10T18:31:57.242 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:31:57.289 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:31:57.323 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:31:57.369 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:31:57.393 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:31:58.163 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:31:58.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:31:58.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:31:58.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:31:58.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:31:58.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:31:58.164 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:31:58.484 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:31:58.484 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:31:58.485 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:31:58.486 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:31:58.486 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:58.486 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:31:58.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:31:58.761 INFO:teuthology.orchestra.run.smithi131.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.5 MB/s | 6.0 MB 00:02 2024-09-10T18:31:58.763 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:58.764 INFO:teuthology.orchestra.run.smithi131.stdout:Total 2.7 MB/s | 7.0 MB 00:02 2024-09-10T18:31:58.838 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:31:58.850 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:31:58.850 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:31:58.951 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:31:58.951 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:31:59.256 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:31:59.314 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:22 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:31:59.417 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:31:59.447 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:31:59.447 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:59.447 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:31:59.447 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:31:59.448 INFO:teuthology.orchestra.run.smithi175.stdout:Install 6 Packages 2024-09-10T18:31:59.449 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:31:59.449 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 7.0 M 2024-09-10T18:31:59.449 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 33 M 2024-09-10T18:31:59.449 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:31:59.471 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:31:59.831 INFO:teuthology.orchestra.run.smithi175.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 550 kB/s | 124 kB 00:00 2024-09-10T18:31:59.882 INFO:teuthology.orchestra.run.smithi175.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-10T18:31:59.907 INFO:teuthology.orchestra.run.smithi175.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-10T18:31:59.932 INFO:teuthology.orchestra.run.smithi175.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T18:31:59.998 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:32:00.091 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:32:00.146 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:32:00.175 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:32:00.240 INFO:teuthology.orchestra.run.smithi175.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 391 kB/s | 248 kB 00:00 2024-09-10T18:32:01.012 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:32:01.012 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:32:01.013 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:32:01.013 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:32:01.013 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:32:01.013 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:32:01.282 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:32:01.282 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:01.283 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:01.551 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-fuse 2024-09-10T18:32:01.907 INFO:teuthology.orchestra.run.smithi175.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.6 MB/s | 6.0 MB 00:02 2024-09-10T18:32:01.909 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:01.910 INFO:teuthology.orchestra.run.smithi175.stdout:Total 2.8 MB/s | 7.0 MB 00:02 2024-09-10T18:32:01.981 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:01.993 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:01.993 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:02.092 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:02.092 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:02.117 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:22 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:02.250 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:Installing dependencies: 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout:Install 2 Packages 2024-09-10T18:32:02.251 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:02.252 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 893 k 2024-09-10T18:32:02.252 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 2.6 M 2024-09-10T18:32:02.252 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:02.404 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:02.497 INFO:teuthology.orchestra.run.smithi131.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 355 kB/s | 80 kB 00:00 2024-09-10T18:32:02.572 INFO:teuthology.orchestra.run.smithi131.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-10T18:32:02.573 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:02.573 INFO:teuthology.orchestra.run.smithi131.stdout:Total 2.7 MB/s | 893 kB 00:00 2024-09-10T18:32:02.580 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:32:02.585 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:02.607 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:02.607 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:02.634 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:32:02.688 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:02.688 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:02.876 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:03.002 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:32:03.064 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:32:03.151 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:32:03.244 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:32:03.308 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:32:03.338 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:32:03.704 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:32:03.704 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:32:03.951 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:32:03.951 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:03.951 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:03.951 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:32:03.951 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:03.951 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:04.076 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:32:04.076 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:32:04.076 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:32:04.076 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:32:04.077 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:32:04.077 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:32:04.220 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install ceph-volume 2024-09-10T18:32:04.345 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:32:04.346 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:32:04.347 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:32:04.347 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:04.347 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:04.628 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-fuse 2024-09-10T18:32:04.804 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:24 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:04.945 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:04.945 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:04.945 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repository Size 2024-09-10T18:32:04.945 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 270 k 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 1.3 M 2024-09-10T18:32:04.946 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:05.178 INFO:teuthology.orchestra.run.smithi131.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.1 MB/s | 270 kB 00:00 2024-09-10T18:32:05.178 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:05.179 INFO:teuthology.orchestra.run.smithi131.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-10T18:32:05.179 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:05.187 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:05.187 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:05.200 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:05.213 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:05.213 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:05.332 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:05.333 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:05.333 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:Install 2 Packages 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 893 k 2024-09-10T18:32:05.334 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 2.6 M 2024-09-10T18:32:05.335 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:05.426 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:05.476 INFO:teuthology.orchestra.run.smithi175.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 659 kB/s | 80 kB 00:00 2024-09-10T18:32:05.558 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:32:05.559 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:32:05.559 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:32:05.559 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:32:05.559 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:05.626 INFO:teuthology.orchestra.run.smithi175.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.9 MB/s | 813 kB 00:00 2024-09-10T18:32:05.626 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:05.627 INFO:teuthology.orchestra.run.smithi175.stdout:Total 3.0 MB/s | 893 kB 00:00 2024-09-10T18:32:05.638 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:05.660 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:05.660 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:05.741 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:05.742 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:05.930 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:06.072 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:32:06.141 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:32:06.573 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:32:06.573 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:06.573 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:06.573 INFO:teuthology.orchestra.run.smithi131.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:32:06.573 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:06.574 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:06.843 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:32:06.843 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:32:06.893 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados-devel 2024-09-10T18:32:07.118 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:32:07.118 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:07.118 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:07.118 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:32:07.118 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:07.118 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:07.423 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-volume 2024-09-10T18:32:07.484 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:27 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:07.618 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:07.618 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 126 k 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 448 k 2024-09-10T18:32:07.619 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:07.834 INFO:teuthology.orchestra.run.smithi131.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 589 kB/s | 126 kB 00:00 2024-09-10T18:32:07.834 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:07.834 INFO:teuthology.orchestra.run.smithi131.stdout:Total 585 kB/s | 126 kB 00:00 2024-09-10T18:32:07.835 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:07.840 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:07.840 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:07.903 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:07.904 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:08.014 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:08.085 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:08.153 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-10T18:32:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:08.153 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:08.153 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 270 k 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 1.3 M 2024-09-10T18:32:08.154 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:08.181 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:08.380 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T18:32:08.381 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:08.381 INFO:teuthology.orchestra.run.smithi175.stdout:Total 1.1 MB/s | 270 kB 00:00 2024-09-10T18:32:08.381 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:08.389 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:08.389 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:08.413 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:08.414 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:08.622 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:08.622 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:08.682 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:32:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:32:08.698 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:32:08.698 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:32:08.698 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:08.899 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:08.900 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:08.900 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:08.900 INFO:teuthology.orchestra.run.smithi131.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:08.900 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:08.900 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:09.170 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs2 2024-09-10T18:32:09.754 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:09.774 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:32:09.774 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:09.774 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:09.774 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:32:09.774 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:09.774 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:09.847 INFO:teuthology.orchestra.run.smithi131.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:09.892 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:09.893 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:09.894 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:09.967 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install libcephfs-devel 2024-09-10T18:32:10.066 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados-devel 2024-09-10T18:32:10.540 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:10.646 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:32:10.673 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:10.674 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:10.674 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:10.674 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:32:10.674 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:10.674 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 25 k 2024-09-10T18:32:10.674 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 110 k 2024-09-10T18:32:10.675 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:10.780 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:10.781 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:32:10.782 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:10.782 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 126 k 2024-09-10T18:32:10.782 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 448 k 2024-09-10T18:32:10.782 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:10.830 INFO:teuthology.orchestra.run.smithi131.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 161 kB/s | 25 kB 00:00 2024-09-10T18:32:10.830 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:10.830 INFO:teuthology.orchestra.run.smithi131.stdout:Total 159 kB/s | 25 kB 00:00 2024-09-10T18:32:10.831 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:10.834 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:10.834 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:10.835 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:10.836 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:10.926 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:10.976 INFO:teuthology.orchestra.run.smithi175.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 648 kB/s | 126 kB 00:00 2024-09-10T18:32:10.977 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:10.977 INFO:teuthology.orchestra.run.smithi175.stdout:Total 643 kB/s | 126 kB 00:00 2024-09-10T18:32:10.977 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:10.983 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:10.983 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:11.045 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:11.046 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:11.052 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:11.213 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:11.307 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:11.423 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:11.656 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:11.656 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:11.656 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:11.656 INFO:teuthology.orchestra.run.smithi131.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:11.656 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:11.656 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:11.757 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:11.839 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librados2 2024-09-10T18:32:12.061 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:12.061 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:12.061 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:12.061 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:12.061 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:12.062 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:12.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs2 2024-09-10T18:32:12.404 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:12.494 INFO:teuthology.orchestra.run.smithi131.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:12.537 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:12.538 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:12.538 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:12.602 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install librbd1 2024-09-10T18:32:12.911 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:13.003 INFO:teuthology.orchestra.run.smithi175.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:13.047 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:13.048 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:13.048 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:13.114 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install libcephfs-devel 2024-09-10T18:32:13.166 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:13.257 INFO:teuthology.orchestra.run.smithi131.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:13.299 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:13.300 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:13.300 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:13.367 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rados 2024-09-10T18:32:13.678 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:13.811 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:13.812 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:32:13.813 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:13.813 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 25 k 2024-09-10T18:32:13.813 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 110 k 2024-09-10T18:32:13.813 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:13.927 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:13.954 INFO:teuthology.orchestra.run.smithi175.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 176 kB/s | 25 kB 00:00 2024-09-10T18:32:13.955 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:13.955 INFO:teuthology.orchestra.run.smithi175.stdout:Total 174 kB/s | 25 kB 00:00 2024-09-10T18:32:13.955 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:13.959 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:13.959 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:13.960 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:13.961 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:14.016 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:14.050 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:14.059 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:14.060 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:14.060 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:14.125 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rgw 2024-09-10T18:32:14.143 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:14.521 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:14.690 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:14.738 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:14.738 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:14.738 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:14.738 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:14.738 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:14.738 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:14.781 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:14.824 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:14.824 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:14.825 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:14.888 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-cephfs 2024-09-10T18:32:14.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librados2 2024-09-10T18:32:15.456 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:15.499 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:15.546 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:15.588 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:15.589 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:15.589 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:15.589 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:15.632 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:15.633 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:15.633 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:15.654 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install python3-rbd 2024-09-10T18:32:15.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install librbd1 2024-09-10T18:32:16.223 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:16.257 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:16.314 INFO:teuthology.orchestra.run.smithi131.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:16.347 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:16.357 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:16.358 INFO:teuthology.orchestra.run.smithi131.stdout:Nothing to do. 2024-09-10T18:32:16.358 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:16.390 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:16.391 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:16.391 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:16.421 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-fuse 2024-09-10T18:32:16.451 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rados 2024-09-10T18:32:16.984 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:17.017 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:17.106 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:17.116 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:32:17.117 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:17.118 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 82 k 2024-09-10T18:32:17.118 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 222 k 2024-09-10T18:32:17.118 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:17.149 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:17.150 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:17.150 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:17.211 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rgw 2024-09-10T18:32:17.308 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 429 kB/s | 82 kB 00:00 2024-09-10T18:32:17.309 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:17.309 INFO:teuthology.orchestra.run.smithi131.stdout:Total 426 kB/s | 82 kB 00:00 2024-09-10T18:32:17.309 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:17.316 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:17.316 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:17.379 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:17.379 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:17.525 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:17.621 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:17.779 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:17.872 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:17.915 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:17.915 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:17.916 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:17.977 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-cephfs 2024-09-10T18:32:18.023 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:18.273 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:18.273 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:18.273 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:18.273 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:18.273 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:18.274 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:18.518 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-mirror 2024-09-10T18:32:18.548 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:18.639 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:18.682 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:18.683 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:18.683 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:18.747 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install python3-rbd 2024-09-10T18:32:19.093 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:19.231 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:19.232 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:19.232 INFO:teuthology.orchestra.run.smithi131.stdout: Package Arch Version Repo Size 2024-09-10T18:32:19.232 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 2.9 M 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 12 M 2024-09-10T18:32:19.233 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:19.305 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:19.396 INFO:teuthology.orchestra.run.smithi175.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:32:19.438 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:19.439 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-09-10T18:32:19.439 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:19.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-fuse 2024-09-10T18:32:19.570 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.7 MB/s | 2.9 MB 00:00 2024-09-10T18:32:19.571 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:19.571 INFO:teuthology.orchestra.run.smithi131.stdout:Total 8.7 MB/s | 2.9 MB 00:00 2024-09-10T18:32:19.571 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:19.580 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:19.580 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:19.662 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:19.663 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:20.066 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:20.085 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:20.146 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:20.171 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:20.171 INFO:teuthology.orchestra.run.smithi131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:32:20.171 INFO:teuthology.orchestra.run.smithi131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:32:20.171 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:32:20.171 INFO:teuthology.orchestra.run.smithi131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:32:20.171 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:20.197 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:20.197 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:32:20.198 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:20.199 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 82 k 2024-09-10T18:32:20.199 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 222 k 2024-09-10T18:32:20.199 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:20.399 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 408 kB/s | 82 kB 00:00 2024-09-10T18:32:20.400 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:20.400 INFO:teuthology.orchestra.run.smithi175.stdout:Total 405 kB/s | 82 kB 00:00 2024-09-10T18:32:20.400 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:20.407 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:20.407 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:20.470 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:20.471 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:20.613 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:20.713 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:21.131 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:21.314 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:21.367 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:21.368 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:21.368 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:21.368 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:21.545 DEBUG:teuthology.orchestra.run.smithi131:> sudo yum -y install rbd-nbd 2024-09-10T18:32:21.633 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-mirror 2024-09-10T18:32:22.118 INFO:teuthology.orchestra.run.smithi131.stdout:Last metadata expiration check: 0:01:42 ago on Tue 10 Sep 2024 06:30:40 PM UTC. 2024-09-10T18:32:22.211 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:45 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:22.252 INFO:teuthology.orchestra.run.smithi131.stdout:Dependencies resolved. 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout: Package Architecture Version Repository Size 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:Installing: 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction Summary 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:================================================================================ 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:Install 1 Package 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:Total download size: 165 k 2024-09-10T18:32:22.253 INFO:teuthology.orchestra.run.smithi131.stdout:Installed size: 477 k 2024-09-10T18:32:22.254 INFO:teuthology.orchestra.run.smithi131.stdout:Downloading Packages: 2024-09-10T18:32:22.350 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:22.350 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:22.350 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repo Size 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 2.9 M 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 12 M 2024-09-10T18:32:22.351 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:22.466 INFO:teuthology.orchestra.run.smithi131.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 775 kB/s | 165 kB 00:00 2024-09-10T18:32:22.467 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:22.467 INFO:teuthology.orchestra.run.smithi131.stdout:Total 771 kB/s | 165 kB 00:00 2024-09-10T18:32:22.467 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction check 2024-09-10T18:32:22.475 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction check succeeded. 2024-09-10T18:32:22.475 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction test 2024-09-10T18:32:22.538 INFO:teuthology.orchestra.run.smithi131.stdout:Transaction test succeeded. 2024-09-10T18:32:22.538 INFO:teuthology.orchestra.run.smithi131.stdout:Running transaction 2024-09-10T18:32:22.677 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 9.0 MB/s | 2.9 MB 00:00 2024-09-10T18:32:22.678 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:22.678 INFO:teuthology.orchestra.run.smithi175.stdout:Total 9.0 MB/s | 2.9 MB 00:00 2024-09-10T18:32:22.678 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:22.687 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:22.687 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:22.699 INFO:teuthology.orchestra.run.smithi131.stdout: Preparing : 1/1 2024-09-10T18:32:22.768 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:22.768 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:22.796 INFO:teuthology.orchestra.run.smithi131.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:23.183 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:23.212 INFO:teuthology.orchestra.run.smithi131.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:23.244 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:23.269 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:23.269 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:32:23.269 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:32:23.269 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:32:23.269 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:32:23.270 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:23.458 INFO:teuthology.orchestra.run.smithi131.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:23.458 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:23.458 INFO:teuthology.orchestra.run.smithi131.stdout:Installed: 2024-09-10T18:32:23.458 INFO:teuthology.orchestra.run.smithi131.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:23.458 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:32:23.459 INFO:teuthology.orchestra.run.smithi131.stdout:Complete! 2024-09-10T18:32:23.679 DEBUG:teuthology.parallel:result is None 2024-09-10T18:32:24.459 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:24.459 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:24.459 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:24.459 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:24.459 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:24.459 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:24.707 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install rbd-nbd 2024-09-10T18:32:25.295 INFO:teuthology.orchestra.run.smithi175.stdout:Last metadata expiration check: 0:01:48 ago on Tue 10 Sep 2024 06:30:37 PM UTC. 2024-09-10T18:32:25.438 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-10T18:32:25.439 INFO:teuthology.orchestra.run.smithi175.stdout:Install 1 Package 2024-09-10T18:32:25.440 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:25.440 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 165 k 2024-09-10T18:32:25.440 INFO:teuthology.orchestra.run.smithi175.stdout:Installed size: 477 k 2024-09-10T18:32:25.440 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-10T18:32:25.656 INFO:teuthology.orchestra.run.smithi175.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 765 kB/s | 165 kB 00:00 2024-09-10T18:32:25.656 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:25.657 INFO:teuthology.orchestra.run.smithi175.stdout:Total 759 kB/s | 165 kB 00:00 2024-09-10T18:32:25.657 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-10T18:32:25.665 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-10T18:32:25.665 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-10T18:32:25.732 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-10T18:32:25.733 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-10T18:32:25.894 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-10T18:32:26.005 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:26.485 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:26.765 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:32:26.765 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:26.765 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-10T18:32:26.765 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:32:26.765 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:32:26.765 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-10T18:32:27.044 DEBUG:teuthology.parallel:result is None 2024-09-10T18:32:27.045 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:32:27.192 DEBUG:teuthology.orchestra.run.smithi131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:32:27.218 INFO:teuthology.orchestra.run.smithi131.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:32:27.219 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:32:27.219 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:32:27.221 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:32:27.362 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:32:27.384 INFO:teuthology.orchestra.run.smithi175.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:32:27.385 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:32:27.385 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:32:27.387 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:32:27.387 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:32:27.387 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:32:27.418 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:32:27.419 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:32:27.451 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:32:27.451 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:32:27.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:32:27.486 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:32:27.553 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:32:27.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:32:27.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:32:27.644 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:32:27.644 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:32:27.644 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:32:27.671 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:32:27.736 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:32:27.736 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:32:27.761 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:32:27.826 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:32:27.827 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:32:27.827 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:32:27.853 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:32:27.919 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:32:27.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:32:27.944 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:32:28.009 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:32:28.099 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'mon_bind_msgr2': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'cephadm-package'} 2024-09-10T18:32:28.099 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:32:28.099 INFO:tasks.cephadm:Cluster fsid is 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:32:28.099 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:32:28.100 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.131:6789]', 'mon.c': '[v1:172.21.15.131:6790]', 'mon.b': '[v1:172.21.15.175:6789]'} 2024-09-10T18:32:28.100 INFO:tasks.cephadm:First mon is mon.a on smithi131 2024-09-10T18:32:28.100 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:32:28.100 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:32:28.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo hostname $(hostname -s) 2024-09-10T18:32:28.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo hostname $(hostname -s) 2024-09-10T18:32:28.161 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:32:28.161 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:32:28.169 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:32:29.763 INFO:teuthology.orchestra.run.smithi175.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:32:29.863 INFO:teuthology.orchestra.run.smithi131.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout: "repo_digests": [ 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout: ] 2024-09-10T18:32:57.798 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-09-10T18:32:58.830 INFO:teuthology.orchestra.run.smithi131.stdout:{ 2024-09-10T18:32:58.831 INFO:teuthology.orchestra.run.smithi131.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:32:58.831 INFO:teuthology.orchestra.run.smithi131.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:32:58.831 INFO:teuthology.orchestra.run.smithi131.stdout: "repo_digests": [ 2024-09-10T18:32:58.831 INFO:teuthology.orchestra.run.smithi131.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:32:58.831 INFO:teuthology.orchestra.run.smithi131.stdout: ] 2024-09-10T18:32:58.831 INFO:teuthology.orchestra.run.smithi131.stdout:} 2024-09-10T18:32:58.851 DEBUG:teuthology.orchestra.run.smithi131:> sudo mkdir -p /etc/ceph 2024-09-10T18:32:58.904 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph 2024-09-10T18:32:58.936 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 777 /etc/ceph 2024-09-10T18:32:58.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /etc/ceph 2024-09-10T18:32:59.005 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:32:59.006 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-10T18:32:59.007 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:32:59.008 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:32:59.008 DEBUG:teuthology.orchestra.run.smithi131:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:32:59.024 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0846fc60-6fa3-11ef-bcea-c7b262605968 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-10T18:32:59.024 DEBUG:teuthology.orchestra.run.smithi131:mon.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:32:59.067 DEBUG:teuthology.orchestra.run.smithi131:mgr.y> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:32:59.109 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:32:59.110 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.131:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:32:59.307 INFO:teuthology.orchestra.run.smithi131.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:32:59.308 INFO:teuthology.orchestra.run.smithi131.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', '0846fc60-6fa3-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.131:6789]', '--skip-admin-label'] 2024-09-10T18:32:59.331 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:32:59.332 INFO:teuthology.orchestra.run.smithi131.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:32:59.332 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying podman|docker is present... 2024-09-10T18:32:59.353 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:32:59.353 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying lvm2 is present... 2024-09-10T18:32:59.354 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying time synchronization is in place... 2024-09-10T18:32:59.362 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:32:59.362 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:32:59.368 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:32:59.368 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-10T18:32:59.375 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-09-10T18:32:59.382 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-09-10T18:32:59.382 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:32:59.382 INFO:teuthology.orchestra.run.smithi131.stdout:Repeating the final host check... 2024-09-10T18:32:59.405 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:32:59.405 INFO:teuthology.orchestra.run.smithi131.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:32:59.405 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl is present 2024-09-10T18:32:59.405 INFO:teuthology.orchestra.run.smithi131.stdout:lvcreate is present 2024-09-10T18:32:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:32:59.412 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:32:59.418 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:32:59.419 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-10T18:32:59.425 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout enabled 2024-09-10T18:32:59.432 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout active 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Host looks OK 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Cluster fsid: 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Acquiring lock 140273592288016 on /run/cephadm/0846fc60-6fa3-11ef-bcea-c7b262605968.lock 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Lock 140273592288016 acquired on /run/cephadm/0846fc60-6fa3-11ef-bcea-c7b262605968.lock 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying IP 172.21.15.131 port 6789 ... 2024-09-10T18:32:59.433 INFO:teuthology.orchestra.run.smithi131.stdout:Base mon IP(s) is [172.21.15.131:6789], mon addrv is [v1:172.21.15.131:6789] 2024-09-10T18:32:59.437 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.131 metric 100 2024-09-10T18:32:59.437 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.131 metric 100 2024-09-10T18:32:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:32:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T18:32:59.440 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T18:32:59.442 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:72f9/64 scope link noprefixroute 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Mon IP `172.21.15.131` is in CIDR network `172.21.0.0/20` 2024-09-10T18:32:59.443 INFO:teuthology.orchestra.run.smithi131.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-09-10T18:32:59.444 INFO:teuthology.orchestra.run.smithi131.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:32:59.444 INFO:teuthology.orchestra.run.smithi131.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:32:59.906 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:32:59.906 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:32:59.906 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:32:59.906 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:32:59.907 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:32:59.907 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:32:59.907 INFO:teuthology.orchestra.run.smithi131.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:33:00.810 INFO:teuthology.orchestra.run.smithi131.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:33:00.810 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:33:00.810 INFO:teuthology.orchestra.run.smithi131.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:33:01.840 INFO:teuthology.orchestra.run.smithi131.stdout:stat: stdout 167 167 2024-09-10T18:33:01.840 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial keys... 2024-09-10T18:33:02.787 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQDekOBm9NEcDRAAwfA6l24f7ad2Qk19bGTL+g== 2024-09-10T18:33:03.792 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQDfkOBm8NH4BxAAAcANPCrjWo/VnAsRGma+1w== 2024-09-10T18:33:04.681 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph-authtool: stdout AQDgkOBmfuvkBhAAsRaDaIyXs4nmf+WdztvGHA== 2024-09-10T18:33:04.681 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial monmap... 2024-09-10T18:33:05.620 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:33:05.620 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:monmaptool for a [v1:172.21.15.131:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:setting min_mon_release = octopus 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: set fsid to 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:33:05.621 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mon... 2024-09-10T18:33:06.559 INFO:teuthology.orchestra.run.smithi131.stdout:create mon.a on 2024-09-10T18:33:06.771 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T18:33:06.906 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:33:07.056 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968.target. 2024-09-10T18:33:07.063 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968.target. 2024-09-10T18:33:07.231 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:33:07.231 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:33:07.391 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968.target.wants/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@.service. 2024-09-10T18:33:07.863 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:07 smithi131 podman[28863]: 2024-09-10 18:33:07.824536292 +0000 UTC m=+0.316973857 container start 0e151f4dea7a04bc689aa6ab582efbe279eec02ee06fbcbfe306b1a6483f0e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T18:33:07.949 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:33:07.950 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-10T18:33:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:33:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-10T18:33:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-10T18:33:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:33:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon to start... 2024-09-10T18:33:07.957 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mon... 2024-09-10T18:33:08.176 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:07 smithi131 ceph-mon[28901]: mkfs 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:08.176 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:07 smithi131 ceph-mon[28901]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:33:08.176 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:07 smithi131 bash[28863]: 0e151f4dea7a04bc689aa6ab582efbe279eec02ee06fbcbfe306b1a6483f0e15 2024-09-10T18:33:08.176 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:07 smithi131 systemd[1]: Started Ceph mon.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:33:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:08 smithi131 ceph-mon[28901]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:33:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:08 smithi131 ceph-mon[28901]: monmap e1: 1 mons at {a=v1:172.21.15.131:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:33:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:08 smithi131 ceph-mon[28901]: fsmap 2024-09-10T18:33:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:08 smithi131 ceph-mon[28901]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:33:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:08 smithi131 ceph-mon[28901]: mgrmap e1: no daemons active 2024-09-10T18:33:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:08 smithi131 ceph-mon[28901]: from='client.? v1:172.21.15.131:0/1171638624' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout id: 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.580204s) 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:15.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.371+0000 7f6b4774c640 1 Processor -- start 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.371+0000 7f6b4774c640 1 -- start start 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.371+0000 7f6b4774c640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6b400ffd70 con 0x7f6b400ff250 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b454e8640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6b400ff250 0x7f6b400ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44742/0 (socket says 172.21.15.131:44742) 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b454e8640 1 -- 172.21.15.131:0/1192526917 learned_addr learned my addr 172.21.15.131:0/1192526917 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2411985453 0 0) 0x7f6b400ffd70 con 0x7f6b400ff250 2024-09-10T18:33:15.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2c0034e0 con 0x7f6b400ff250 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1499335124 0 0) 0x7f6b2c0034e0 con 0x7f6b400ff250 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40104760 con 0x7f6b400ff250 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f6b3c001590 con 0x7f6b400ff250 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6b3c0018f0 con 0x7f6b400ff250 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.372+0000 7f6b37fff640 1 -- 172.21.15.131:0/1192526917 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f6b3c001c40 con 0x7f6b400ff250 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 -- 172.21.15.131:0/1192526917 >> v1:172.21.15.131:6789/0 conn(0x7f6b400ff250 legacy=0x7f6b400ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 -- 172.21.15.131:0/1192526917 shutdown_connections 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 -- 172.21.15.131:0/1192526917 >> 172.21.15.131:0/1192526917 conn(0x7f6b400faa90 msgr2=0x7f6b400fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:15.125 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 -- 172.21.15.131:0/1192526917 shutdown_connections 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 -- 172.21.15.131:0/1192526917 wait complete. 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 Processor -- start 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.373+0000 7f6b4774c640 1 -- start start 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b4774c640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6b401122d0 con 0x7f6b400ff250 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b454e8640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f6b400ff250 0x7f6b400741f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44758/0 (socket says 172.21.15.131:44758) 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b454e8640 1 -- 172.21.15.131:0/1171638624 learned_addr learned my addr 172.21.15.131:0/1171638624 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940435906 0 0) 0x7f6b401122d0 con 0x7f6b400ff250 2024-09-10T18:33:15.126 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b2c003d20 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3761372643 0 0) 0x7f6b2c003d20 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b401122d0 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f6b3c001b80 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4191864493 0 0) 0x7f6b401122d0 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b40113470 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b40112460 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f6b3c004ce0 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f6b3c004ff0 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f6b3c005450 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.374+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b40112930 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.375+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f6b3c005c70 con 0x7f6b400ff250 2024-09-10T18:33:15.127 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.375+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b40103890 con 0x7f6b400ff250 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.377+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f6b3c006ac0 con 0x7f6b400ff250 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.436+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f6b401073b0 con 0x7f6b400ff250 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.436+0000 7f6b35ffb640 1 -- 172.21.15.131:0/1171638624 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 3971742917) 0x7f6b3c006180 con 0x7f6b400ff250 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.437+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 >> v1:172.21.15.131:6789/0 conn(0x7f6b400ff250 legacy=0x7f6b400741f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.437+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 shutdown_connections 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.437+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 >> 172.21.15.131:0/1171638624 conn(0x7f6b400faa90 msgr2=0x7f6b40104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.437+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 shutdown_connections 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:08.437+0000 7f6b4774c640 1 -- 172.21.15.131:0/1171638624 wait complete. 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:mon is available 2024-09-10T18:33:15.128 INFO:teuthology.orchestra.run.smithi131.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:33:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:18 smithi131 ceph-mon[28901]: from='client.? v1:172.21.15.131:0/3555626503' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:33:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:18 smithi131 ceph-mon[28901]: from='client.? v1:172.21.15.131:0/3555626503' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:33:19.936 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:19.936 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:33:19.936 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:19.936 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:33:19.936 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.131:6789] 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:33:19.937 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.176+0000 7fd8924b2640 1 Processor -- start 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.176+0000 7fd8924b2640 1 -- start start 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd8924b2640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd88c1046c0 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88bfff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd88c073c10 0x7fd88c102fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35038/0 (socket says 172.21.15.131:35038) 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88bfff640 1 -- 172.21.15.131:0/3450686204 learned_addr learned my addr 172.21.15.131:0/3450686204 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137411580 0 0) 0x7fd88c1046c0 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8740034e0 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4226599428 0 0) 0x7fd8740034e0 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c105860 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.177+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd87c001590 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd87c0018f0 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd88affd640 1 -- 172.21.15.131:0/3450686204 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd87c001c40 con 0x7fd88c073c10 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd8924b2640 1 -- 172.21.15.131:0/3450686204 >> v1:172.21.15.131:6789/0 conn(0x7fd88c073c10 legacy=0x7fd88c102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:19.938 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd8924b2640 1 -- 172.21.15.131:0/3450686204 shutdown_connections 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd8924b2640 1 -- 172.21.15.131:0/3450686204 >> 172.21.15.131:0/3450686204 conn(0x7fd88c0fa9d0 msgr2=0x7fd88c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd8924b2640 1 -- 172.21.15.131:0/3450686204 shutdown_connections 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd8924b2640 1 -- 172.21.15.131:0/3450686204 wait complete. 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.178+0000 7fd8924b2640 1 Processor -- start 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd8924b2640 1 -- start start 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd8924b2640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd88c10ec00 con 0x7fd88c073c10 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd88bfff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd88c073c10 0x7fd88c10e4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:35052/0 (socket says 172.21.15.131:35052) 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd88bfff640 1 -- 172.21.15.131:0/3555626503 learned_addr learned my addr 172.21.15.131:0/3555626503 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204336641 0 0) 0x7fd88c10ec00 con 0x7fd88c073c10 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd874003d20 con 0x7fd88c073c10 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1282580772 0 0) 0x7fd874003d20 con 0x7fd88c073c10 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd88c10ec00 con 0x7fd88c073c10 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.179+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd87c001550 con 0x7fd88c073c10 2024-09-10T18:33:19.939 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2394666463 0 0) 0x7fd88c10ec00 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c10ed90 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd88c10f060 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88c112b40 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fd87c004c70 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd87c004f80 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fd87c0053e0 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.180+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7fd87c0063b0 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.181+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd878005040 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.183+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fd87c0065f0 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.238+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd878005870 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.241+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 10 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fd87c005c00 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.241+0000 7fd888ff9640 1 -- 172.21.15.131:0/3555626503 <== mon.0 v1:172.21.15.131:6789/0 11 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+354 (unknown 1213389831 0 587506351) 0x7fd87c007450 con 0x7fd88c073c10 2024-09-10T18:33:19.940 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.242+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 >> v1:172.21.15.131:6789/0 conn(0x7fd88c073c10 legacy=0x7fd88c10e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:19.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.242+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 shutdown_connections 2024-09-10T18:33:19.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.242+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 >> 172.21.15.131:0/3555626503 conn(0x7fd88c0fa9d0 msgr2=0x7fd88c1059f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:19.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.242+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 shutdown_connections 2024-09-10T18:33:19.941 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:18.242+0000 7fd8924b2640 1 -- 172.21.15.131:0/3555626503 wait complete. 2024-09-10T18:33:19.944 INFO:teuthology.orchestra.run.smithi131.stdout:Generating new minimal ceph.conf... 2024-09-10T18:33:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:20 smithi131 ceph-mon[28901]: from='client.? v1:172.21.15.131:0/162713723' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:33:20.948 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.307+0000 7f7da4c07640 1 Processor -- start 2024-09-10T18:33:20.948 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.307+0000 7f7da4c07640 1 -- start start 2024-09-10T18:33:20.948 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.307+0000 7f7da4c07640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7da00ffd70 con 0x7f7da00ff250 2024-09-10T18:33:20.948 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.307+0000 7f7d9e59c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7da00ff250 0x7f7da00ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43604/0 (socket says 172.21.15.131:43604) 2024-09-10T18:33:20.948 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.307+0000 7f7d9e59c640 1 -- 172.21.15.131:0/1777087563 learned_addr learned my addr 172.21.15.131:0/1777087563 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:20.948 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072532829 0 0) 0x7f7da00ffd70 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d8c0034e0 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1425864031 0 0) 0x7f7d8c0034e0 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da0104710 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7d98001590 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f7d98001cb0 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.308+0000 7f7d9d59a640 1 -- 172.21.15.131:0/1777087563 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7d98005670 con 0x7f7da00ff250 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- 172.21.15.131:0/1777087563 >> v1:172.21.15.131:6789/0 conn(0x7f7da00ff250 legacy=0x7f7da00ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- 172.21.15.131:0/1777087563 shutdown_connections 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- 172.21.15.131:0/1777087563 >> 172.21.15.131:0/1777087563 conn(0x7f7da00fa9d0 msgr2=0x7f7da00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- 172.21.15.131:0/1777087563 shutdown_connections 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- 172.21.15.131:0/1777087563 wait complete. 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 Processor -- start 2024-09-10T18:33:20.949 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- start start 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.309+0000 7f7da4c07640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7da010f300 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d9e59c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7da00ff250 0x7f7da010ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43616/0 (socket says 172.21.15.131:43616) 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d9e59c640 1 -- 172.21.15.131:0/162713723 learned_addr learned my addr 172.21.15.131:0/162713723 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2993752615 0 0) 0x7f7da010f300 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d8c0053f0 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 570244016 0 0) 0x7f7d8c0053f0 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7da010f300 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7d980047a0 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2799058170 0 0) 0x7f7da010f300 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da010f490 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da010f760 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da0113240 con 0x7f7da00ff250 2024-09-10T18:33:20.950 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f7d98004ba0 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7d98006090 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f7d98006570 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.310+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f7d98007630 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.311+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d80005040 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.313+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f7d98004d10 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.368+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7d800052f0 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.369+0000 7f7d96ffd640 1 -- 172.21.15.131:0/162713723 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+148 (unknown 1452402520 0 3472189743) 0x7f7d980059a0 con 0x7f7da00ff250 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.369+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 >> v1:172.21.15.131:6789/0 conn(0x7f7da00ff250 legacy=0x7f7da010ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.370+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 shutdown_connections 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.370+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 >> 172.21.15.131:0/162713723 conn(0x7f7da00fa9d0 msgr2=0x7f7da0072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.370+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 shutdown_connections 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:20.370+0000 7f7da4c07640 1 -- 172.21.15.131:0/162713723 wait complete. 2024-09-10T18:33:20.951 INFO:teuthology.orchestra.run.smithi131.stdout:Restarting the monitor... 2024-09-10T18:33:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:20 smithi131 systemd[1]: Stopping Ceph mon.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:33:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:21 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a[28878]: 2024-09-10T18:33:21.211+0000 7f5d9146c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:33:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:21 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a[28878]: 2024-09-10T18:33:21.211+0000 7f5d9146c640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:33:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:21 smithi131 podman[29217]: 2024-09-10 18:33:21.365516652 +0000 UTC m=+0.250703055 container died 0e151f4dea7a04bc689aa6ab582efbe279eec02ee06fbcbfe306b1a6483f0e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T18:33:21.988 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:21 smithi131 podman[29217]: 2024-09-10 18:33:21.734625376 +0000 UTC m=+0.619811781 container cleanup 0e151f4dea7a04bc689aa6ab582efbe279eec02ee06fbcbfe306b1a6483f0e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:33:21.988 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:21 smithi131 bash[29217]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:33:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:21 smithi131 podman[29234]: 2024-09-10 18:33:21.985594931 +0000 UTC m=+0.613111184 container remove 0e151f4dea7a04bc689aa6ab582efbe279eec02ee06fbcbfe306b1a6483f0e15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T18:33:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:33:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 systemd[1]: Stopped Ceph mon.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:33:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 systemd[1]: Starting Ceph mon.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:33:22.596 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 podman[29331]: 2024-09-10 18:33:22.344748647 +0000 UTC m=+0.106558608 container create e35573a25943145ac0777f4f0facf5fe0e3e19af402f02b277b94ac5b9fd912d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 podman[29331]: 2024-09-10 18:33:22.253719303 +0000 UTC m=+0.015529218 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 podman[29331]: 2024-09-10 18:33:22.563796393 +0000 UTC m=+0.325606307 container init e35573a25943145ac0777f4f0facf5fe0e3e19af402f02b277b94ac5b9fd912d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 podman[29331]: 2024-09-10 18:33:22.568532653 +0000 UTC m=+0.330342569 container start e35573a25943145ac0777f4f0facf5fe0e3e19af402f02b277b94ac5b9fd912d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: pidfile_write: ignore empty --pid-file 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: load: jerasure load: lrc 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:33:22.597 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: DB SUMMARY 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: DB Session ID: UQPURJOCXQ1GJ8L4GY2K 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 83798 ; 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:33:22.598 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.env: 0x5598b6e49440 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.fs: Posix File System 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.info_log: 0x5598b7a4ee60 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.statistics: (nil) 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.use_fsync: 0 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:33:22.599 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.db_log_dir: 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:33:22.600 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.write_buffer_manager: 0x5598b7b0c780 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:33:22.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:33:22.602 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:33:22.602 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:33:22.602 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:33:22.603 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:33:22.603 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:33:22.603 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:33:22.603 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.unordered_write: 0 2024-09-10T18:33:22.603 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.row_cache: None 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.wal_filter: None 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:33:22.604 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:33:22.605 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_open_files: -1 2024-09-10T18:33:22.606 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Compression algorithms supported: 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kZSTD supported: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:33:22.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.merge_operator: 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_filter: None 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:33:22.608 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5598b79f5a38) 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_type: 0 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_index_type: 0 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_shortening: 1 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: hash_index_allow_collision: 1 2024-09-10T18:33:22.609 INFO:journalctl@ceph.mon.a.smithi131.stdout: checksum: 1 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: no_block_cache: 0 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache: 0x5598b7a2b1f0 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_options: 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: capacity : 536870912 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: num_shard_bits : 4 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: strict_capacity_limit : 0 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_cache_compressed: (nil) 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: persistent_cache: (nil) 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size: 4096 2024-09-10T18:33:22.610 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_size_deviation: 10 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_restart_interval: 16 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: index_block_restart_interval: 1 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: metadata_block_size: 4096 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: partition_filters: 0 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: use_delta_encoding: 1 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: whole_key_filtering: 1 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: verify_compression: 0 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:33:22.611 INFO:journalctl@ceph.mon.a.smithi131.stdout: format_version: 4 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_index_compression: 1 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout: block_align: 0 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression: NoCompression 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.num_levels: 7 2024-09-10T18:33:22.612 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:33:22.613 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:33:22.614 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:33:22.615 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:33:22.616 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:33:22.617 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.ttl: 2592000 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:33:22.618 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993202591953, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993202592686, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80180, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 78446, "index_size": 196, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10435, "raw_average_key_size": 47, "raw_value_size": 72555, "raw_average_value_size": 331, "num_data_blocks": 10, "num_entries": 219, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993202, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "b8437424-0981-4049-8a85-712c2cc0afa0", "db_session_id": "UQPURJOCXQ1GJ8L4GY2K"}} 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993202593033, "job": 1, "event": "recovery_finished"} 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5598b7a5e700 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: DB pointer 0x5598b7a2e000 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:33:22.619 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** DB Stats ** 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:33:22.620 INFO:journalctl@ceph.mon.a.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 80.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 80.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:33:22.621 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 24.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 24.93 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:33:22.622 INFO:journalctl@ceph.mon.a.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: L0 2/0 80.05 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: Sum 2/0 80.05 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 158.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:33:22.623 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: Cumulative compaction: 0.00 GB write, 24.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: starting mon.a rank 0 at public addrs v1:172.21.15.131:6789/0 at bind addrs v1:172.21.15.131:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???) e1 preinit fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:22.624 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).mds e1 new map 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout: e1 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout: legacy client fscid: -1 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout: 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout: No filesystems configured 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:33:22.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:33:22.689 INFO:teuthology.orchestra.run.smithi131.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:33:22.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:33:22.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: monmap e1: 1 mons at {a=v1:172.21.15.131:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:33:22.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: fsmap 2024-09-10T18:33:22.909 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:33:22.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 ceph-mon[29369]: mgrmap e1: no daemons active 2024-09-10T18:33:22.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 bash[29331]: e35573a25943145ac0777f4f0facf5fe0e3e19af402f02b277b94ac5b9fd912d 2024-09-10T18:33:22.910 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:22 smithi131 systemd[1]: Started Ceph mon.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.161+0000 7fa502e70640 1 Processor -- start 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.161+0000 7fa502e70640 1 -- start start 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa502e70640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa4fc0ffd40 con 0x7fa4fc0ff220 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa500c0c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa4fc0ff220 0x7fa4fc0ff630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43622/0 (socket says 172.21.15.131:43622) 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa500c0c640 1 -- 172.21.15.131:0/2450288911 learned_addr learned my addr 172.21.15.131:0/2450288911 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819715681 0 0) 0x7fa4fc0ffd40 con 0x7fa4fc0ff220 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e80034e0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 631043799 0 0) 0x7fa4e80034e0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc104730 con 0x7fa4fc0ff220 2024-09-10T18:33:23.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.162+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa4ec001910 con 0x7fa4fc0ff220 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fa4ec002030 con 0x7fa4fc0ff220 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa4fb7fe640 1 -- 172.21.15.131:0/2450288911 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa4ec0059f0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa502e70640 1 -- 172.21.15.131:0/2450288911 >> v1:172.21.15.131:6789/0 conn(0x7fa4fc0ff220 legacy=0x7fa4fc0ff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa502e70640 1 -- 172.21.15.131:0/2450288911 shutdown_connections 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa502e70640 1 -- 172.21.15.131:0/2450288911 >> 172.21.15.131:0/2450288911 conn(0x7fa4fc0faaa0 msgr2=0x7fa4fc0fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa502e70640 1 -- 172.21.15.131:0/2450288911 shutdown_connections 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.163+0000 7fa502e70640 1 -- 172.21.15.131:0/2450288911 wait complete. 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa502e70640 1 Processor -- start 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa502e70640 1 -- start start 2024-09-10T18:33:23.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa502e70640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa4fc112ab0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa500c0c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa4fc0ff220 0x7fa4fc0741a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43628/0 (socket says 172.21.15.131:43628) 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa500c0c640 1 -- 172.21.15.131:0/3244888338 learned_addr learned my addr 172.21.15.131:0/3244888338 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981055977 0 0) 0x7fa4fc112ab0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4e80053f0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3849127122 0 0) 0x7fa4e80053f0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa4fc112ab0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.164+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa4ec0013e0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1937691802 0 0) 0x7fa4fc112ab0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4fc113c50 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc112c40 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc113110 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fa4ec004f20 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa4ec006500 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7fa4ec0069e0 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.165+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7fa4ec007a00 con 0x7fa4fc0ff220 2024-09-10T18:33:23.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.166+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4dc005040 con 0x7fa4fc0ff220 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.168+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fa4ec005090 con 0x7fa4fc0ff220 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.223+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 --> v1:172.21.15.131:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fa4dc005870 con 0x7fa4fc0ff220 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.225+0000 7fa4f97fa640 1 -- 172.21.15.131:0/3244888338 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (unknown 534127980 0 0) 0x7fa4ec005e20 con 0x7fa4fc0ff220 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.226+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 >> v1:172.21.15.131:6789/0 conn(0x7fa4fc0ff220 legacy=0x7fa4fc0741a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.226+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 shutdown_connections 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.226+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 >> 172.21.15.131:0/3244888338 conn(0x7fa4fc0faaa0 msgr2=0x7fa4fc102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.226+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 shutdown_connections 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:23.226+0000 7fa502e70640 1 -- 172.21.15.131:0/3244888338 wait complete. 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:Creating mgr... 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:33:23.807 INFO:teuthology.orchestra.run.smithi131.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:33:23.973 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:33:23.974 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:33:24.122 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968.target.wants/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@.service. 2024-09-10T18:33:24.474 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3244888338' entity='client.admin' 2024-09-10T18:33:24.474 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 podman[29606]: 2024-09-10 18:33:24.362922421 +0000 UTC m=+0.119287151 container create 112b87f9c85b1a1507dc6d6c0aedff8b91d3492e18507602f3c945334dbcad36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-10T18:33:24.697 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:33:24.697 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-10T18:33:24.704 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:33:24.704 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-10T18:33:24.704 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-10T18:33:24.704 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:33:24.713 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:33:24.713 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-10T18:33:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:33:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-10T18:33:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-10T18:33:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:33:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr to start... 2024-09-10T18:33:24.721 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr... 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 podman[29606]: 2024-09-10 18:33:24.573988486 +0000 UTC m=+0.330353217 container init 112b87f9c85b1a1507dc6d6c0aedff8b91d3492e18507602f3c945334dbcad36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 podman[29606]: 2024-09-10 18:33:24.578921409 +0000 UTC m=+0.335286140 container start 112b87f9c85b1a1507dc6d6c0aedff8b91d3492e18507602f3c945334dbcad36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 bash[29606]: 112b87f9c85b1a1507dc6d6c0aedff8b91d3492e18507602f3c945334dbcad36 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 systemd[1]: Started Ceph mgr.y for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:24.689+0000 7f0da3938ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:24.749+0000 7f0da3938ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:33:24.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:24.815+0000 7f0da3938ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:33:25.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:24 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:24.940+0000 7f0da3938ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:33:25.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:25 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:25.075+0000 7f0da3938ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:33:25.730 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:25 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:25.335+0000 7f0da3938ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:33:25.730 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:25 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:25.394+0000 7f0da3938ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:33:25.731 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:25 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:25.557+0000 7f0da3938ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:33:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/449993901' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:33:26.753 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:26 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:26.423+0000 7f0da3938ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:33:26.753 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:26 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:26.490+0000 7f0da3938ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:33:26.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:26.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:33:26.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "0846fc60-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:33:26.757 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-10T18:33:26.758 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:33:26.759 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:33:26.767 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:33:26.768 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:33:07.858153+0000", 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.059+0000 7f4cf1b94640 1 Processor -- start 2024-09-10T18:33:26.769 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.059+0000 7f4cf1b94640 1 -- start start 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.059+0000 7f4cf1b94640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cec0769b0 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4ceb7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4cec0765a0 0x7f4cec074b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43656/0 (socket says 172.21.15.131:43656) 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4ceb7fe640 1 -- 172.21.15.131:0/813107139 learned_addr learned my addr 172.21.15.131:0/813107139 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900801797 0 0) 0x7f4cec0769b0 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd40034e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597564946 0 0) 0x7f4cd40034e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec075230 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4cdc001590 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f4cdc001be0 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.060+0000 7f4cea7fc640 1 -- 172.21.15.131:0/813107139 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4cdc0055b0 con 0x7f4cec0765a0 2024-09-10T18:33:26.770 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.061+0000 7f4cf1b94640 1 -- 172.21.15.131:0/813107139 >> v1:172.21.15.131:6789/0 conn(0x7f4cec0765a0 legacy=0x7f4cec074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.061+0000 7f4cf1b94640 1 -- 172.21.15.131:0/813107139 shutdown_connections 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.061+0000 7f4cf1b94640 1 -- 172.21.15.131:0/813107139 >> 172.21.15.131:0/813107139 conn(0x7f4cec0fbb70 msgr2=0x7f4cec0fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.061+0000 7f4cf1b94640 1 -- 172.21.15.131:0/813107139 shutdown_connections 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.061+0000 7f4cf1b94640 1 -- 172.21.15.131:0/813107139 wait complete. 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.061+0000 7f4cf1b94640 1 Processor -- start 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cf1b94640 1 -- start start 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cf1b94640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cec1115e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4ceb7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4cec0765a0 0x7f4cec110ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:43670/0 (socket says 172.21.15.131:43670) 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4ceb7fe640 1 -- 172.21.15.131:0/449993901 learned_addr learned my addr 172.21.15.131:0/449993901 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505119538 0 0) 0x7f4cec1115e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cd4005440 con 0x7f4cec0765a0 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 349605717 0 0) 0x7f4cd4005440 con 0x7f4cec0765a0 2024-09-10T18:33:26.771 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4cec1115e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.062+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4cdc001d50 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1457957254 0 0) 0x7f4cec1115e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec111770 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cec111a40 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cec115520 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f4cdc0017e0 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4cdc005f60 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 1) v1 ==== 529+0+0 (unknown 3533299129 0 0) 0x7f4cdc006400 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.063+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f4cdc007420 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.064+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd8005040 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.067+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7f4cdc001950 con 0x7f4cec0765a0 2024-09-10T18:33:26.772 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.126+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4cd80052f0 con 0x7f4cec0765a0 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.127+0000 7f4cd3fff640 1 -- 172.21.15.131:0/449993901 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (unknown 4201413639 0 3280418151) 0x7f4cdc0058f0 con 0x7f4cec0765a0 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.128+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 >> v1:172.21.15.131:6789/0 conn(0x7f4cec0765a0 legacy=0x7f4cec110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.128+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 shutdown_connections 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.128+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 >> 172.21.15.131:0/449993901 conn(0x7f4cec0fbb70 msgr2=0x7f4cec104c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.128+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 shutdown_connections 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:26.128+0000 7f4cf1b94640 1 -- 172.21.15.131:0/449993901 wait complete. 2024-09-10T18:33:26.773 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:33:27.431 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.199+0000 7f0da3938ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:33:27.431 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.311+0000 7f0da3938ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:33:27.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.429+0000 7f0da3938ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:33:27.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.488+0000 7f0da3938ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:33:27.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.612+0000 7f0da3938ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:33:27.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.677+0000 7f0da3938ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:33:27.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.735+0000 7f0da3938ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:33:27.997 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.867+0000 7f0da3938ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:33:27.997 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.931+0000 7f0da3938ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:33:27.997 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:27 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:27.994+0000 7f0da3938ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:33:28.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:28 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:28.074+0000 7f0da3938ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:33:28.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:28 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:28.134+0000 7f0da3938ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:33:28.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:28 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:28.194+0000 7f0da3938ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:33:29.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: Activating manager daemon y 2024-09-10T18:33:29.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: mgrmap e2: y(active, starting, since 0.00297819s) 2024-09-10T18:33:29.172 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: Manager daemon y is now available 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' 2024-09-10T18:33:29.173 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:29 smithi131 ceph-mon[29369]: from='mgr.14100 v1:172.21.15.131:0/3977946916' entity='mgr.y' 2024-09-10T18:33:29.174 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:28 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:28.972+0000 7f0da3938ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:33:29.964 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:29.964 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:33:29.964 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "0846fc60-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:33:29.964 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:33:29.964 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:29.965 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:33:29.966 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:33:29.974 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:33:29.975 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:33:07.858153+0000", 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:33:29.976 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.245+0000 7fe9ea759640 1 Processor -- start 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.245+0000 7fe9ea759640 1 -- start start 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.245+0000 7fe9ea759640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe9e41046c0 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e3fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe9e4073c10 0x7fe9e4102fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44888/0 (socket says 172.21.15.131:44888) 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e3fff640 1 -- 172.21.15.131:0/109479761 learned_addr learned my addr 172.21.15.131:0/109479761 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757318627 0 0) 0x7fe9e41046c0 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc0034e0 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858330117 0 0) 0x7fe9cc0034e0 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4105860 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe9d4001590 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fe9d4001cb0 con 0x7fe9e4073c10 2024-09-10T18:33:29.977 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.246+0000 7fe9e2ffd640 1 -- 172.21.15.131:0/109479761 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe9d4005670 con 0x7fe9e4073c10 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.247+0000 7fe9ea759640 1 -- 172.21.15.131:0/109479761 >> v1:172.21.15.131:6789/0 conn(0x7fe9e4073c10 legacy=0x7fe9e4102fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.247+0000 7fe9ea759640 1 -- 172.21.15.131:0/109479761 shutdown_connections 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.247+0000 7fe9ea759640 1 -- 172.21.15.131:0/109479761 >> 172.21.15.131:0/109479761 conn(0x7fe9e40fa9d0 msgr2=0x7fe9e40fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.247+0000 7fe9ea759640 1 -- 172.21.15.131:0/109479761 shutdown_connections 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.247+0000 7fe9ea759640 1 -- 172.21.15.131:0/109479761 wait complete. 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.247+0000 7fe9ea759640 1 Processor -- start 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9ea759640 1 -- start start 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9ea759640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe9e4100e00 con 0x7fe9e4073c10 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e3fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe9e4073c10 0x7fe9e41006f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44900/0 (socket says 172.21.15.131:44900) 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e3fff640 1 -- 172.21.15.131:0/1086123629 learned_addr learned my addr 172.21.15.131:0/1086123629 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505433982 0 0) 0x7fe9e4100e00 con 0x7fe9e4073c10 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9cc0053f0 con 0x7fe9e4073c10 2024-09-10T18:33:29.978 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1051543345 0 0) 0x7fe9cc0053f0 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9e4100e00 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe9d40047a0 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 175664782 0 0) 0x7fe9e4100e00 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.248+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4115d50 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.249+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9e4114da0 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.249+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9e4115270 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.249+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fe9d4004ba0 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.249+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe9d4006180 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.249+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 2) v1 ==== 45248+0+0 (unknown 1754963411 0 0) 0x7fe9d40073f0 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.250+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7fe9d40474b0 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.250+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9d0005040 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.252+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (unknown 1092875540 0 1542132831) 0x7fe9d4008160 con 0x7fe9e4073c10 2024-09-10T18:33:29.979 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.336+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe9d00052f0 con 0x7fe9e4073c10 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.336+0000 7fe9e0ff9640 1 -- 172.21.15.131:0/1086123629 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (unknown 4201413639 0 3560056411) 0x7fe9d40163c0 con 0x7fe9e4073c10 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.337+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 >> v1:172.21.15.131:6789/0 conn(0x7fe9e4073c10 legacy=0x7fe9e41006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.337+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 shutdown_connections 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.337+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 >> 172.21.15.131:0/1086123629 conn(0x7fe9e40fa9d0 msgr2=0x7fe9e41028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.337+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 shutdown_connections 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:29.337+0000 7fe9ea759640 1 -- 172.21.15.131:0/1086123629 wait complete. 2024-09-10T18:33:29.980 INFO:teuthology.orchestra.run.smithi131.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:33:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1086123629' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:33:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:30 smithi131 ceph-mon[29369]: mgrmap e3: y(active, since 1.00576s) 2024-09-10T18:33:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:32 smithi131 ceph-mon[29369]: mgrmap e4: y(active, since 2s) 2024-09-10T18:33:33.292 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2433927918' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:33:33.296 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:33.296 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsid": "0846fc60-6fa3-11ef-bcea-c7b262605968", 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:33:33.297 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:33:33.298 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:33:33.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:33:33.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:33:33.306 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.307 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:33:33.308 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:33:07.858153+0000", 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.431+0000 7f3f9bf0a640 1 Processor -- start 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.431+0000 7f3f9bf0a640 1 -- start start 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.431+0000 7f3f9bf0a640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3f94100e90 con 0x7f3f94100370 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.431+0000 7f3f99ca6640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3f94100370 0x7f3f94100780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44902/0 (socket says 172.21.15.131:44902) 2024-09-10T18:33:33.309 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.431+0000 7f3f99ca6640 1 -- 172.21.15.131:0/1699813118 learned_addr learned my addr 172.21.15.131:0/1699813118 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884128917 0 0) 0x7f3f94100e90 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f800034e0 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2573034870 0 0) 0x7f3f800034e0 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94105880 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f3f90001590 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f3f90001cb0 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.432+0000 7f3f98ca4640 1 -- 172.21.15.131:0/1699813118 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f3f90005670 con 0x7f3f94100370 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/1699813118 >> v1:172.21.15.131:6789/0 conn(0x7f3f94100370 legacy=0x7f3f94100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/1699813118 shutdown_connections 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/1699813118 >> 172.21.15.131:0/1699813118 conn(0x7f3f940fbb90 msgr2=0x7f3f940fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/1699813118 shutdown_connections 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/1699813118 wait complete. 2024-09-10T18:33:33.310 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 Processor -- start 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- start start 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f9bf0a640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3f9410f2f0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f99ca6640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3f94100370 0x7f3f9410ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44916/0 (socket says 172.21.15.131:44916) 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.433+0000 7f3f99ca6640 1 -- 172.21.15.131:0/2433927918 learned_addr learned my addr 172.21.15.131:0/2433927918 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2837757536 0 0) 0x7f3f9410f2f0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f800053f0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 673066772 0 0) 0x7f3f800053f0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f9410f2f0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f3f900047a0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1250836371 0 0) 0x7f3f9410f2f0 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9410f480 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9410f750 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f94113230 con 0x7f3f94100370 2024-09-10T18:33:33.311 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f3f90004ba0 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.434+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f3f90006110 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.435+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (unknown 3805857646 0 0) 0x7f3f900114c0 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.435+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f78005040 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.435+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f3f90047980 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.440+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f3f90016c20 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.653+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3f780052f0 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.653+0000 7f3f8a7fc640 1 -- 172.21.15.131:0/2433927918 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (unknown 4201413639 0 1837596824) 0x7f3f90016620 con 0x7f3f94100370 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.655+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 >> v1:172.21.15.131:6800/1625784054 conn(0x7f3f8003a5c0 legacy=0x7f3f8003ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.655+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 >> v1:172.21.15.131:6789/0 conn(0x7f3f94100370 legacy=0x7f3f9410ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.655+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 shutdown_connections 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.656+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 >> 172.21.15.131:0/2433927918 conn(0x7f3f940fbb90 msgr2=0x7f3f940fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.656+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 shutdown_connections 2024-09-10T18:33:33.312 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:32.656+0000 7f3f9bf0a640 1 -- 172.21.15.131:0/2433927918 wait complete. 2024-09-10T18:33:33.313 INFO:teuthology.orchestra.run.smithi131.stdout:mgr is available 2024-09-10T18:33:34.214 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4103346190' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout fsid = 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:33:34.515 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.741+0000 7fa5b0cab640 1 Processor -- start 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.741+0000 7fa5b0cab640 1 -- start start 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5b0cab640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5ac100e90 con 0x7fa5ac100370 2024-09-10T18:33:34.516 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5aa59c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa5ac100370 0x7fa5ac100780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44922/0 (socket says 172.21.15.131:44922) 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5aa59c640 1 -- 172.21.15.131:0/3285188671 learned_addr learned my addr 172.21.15.131:0/3285188671 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548487005 0 0) 0x7fa5ac100e90 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5980034e0 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2425297524 0 0) 0x7fa5980034e0 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac105880 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa59c001590 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fa59c001cb0 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.742+0000 7fa5a959a640 1 -- 172.21.15.131:0/3285188671 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa59c005670 con 0x7fa5ac100370 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 -- 172.21.15.131:0/3285188671 >> v1:172.21.15.131:6789/0 conn(0x7fa5ac100370 legacy=0x7fa5ac100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 -- 172.21.15.131:0/3285188671 shutdown_connections 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 -- 172.21.15.131:0/3285188671 >> 172.21.15.131:0/3285188671 conn(0x7fa5ac0fbb90 msgr2=0x7fa5ac0fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 -- 172.21.15.131:0/3285188671 shutdown_connections 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 -- 172.21.15.131:0/3285188671 wait complete. 2024-09-10T18:33:34.517 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 Processor -- start 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.743+0000 7fa5b0cab640 1 -- start start 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5b0cab640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5ac10f2f0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5aa59c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa5ac100370 0x7fa5ac10ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44932/0 (socket says 172.21.15.131:44932) 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5aa59c640 1 -- 172.21.15.131:0/4103346190 learned_addr learned my addr 172.21.15.131:0/4103346190 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953262543 0 0) 0x7fa5ac10f2f0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5980053f0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3381374933 0 0) 0x7fa5980053f0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5ac10f2f0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa59c0047a0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 572305720 0 0) 0x7fa5ac10f2f0 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac10f480 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac10f750 con 0x7fa5ac100370 2024-09-10T18:33:34.518 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac113230 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fa59c004ba0 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.744+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa59c006110 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.745+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (unknown 3805857646 0 0) 0x7fa59c011400 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.745+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c005040 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.746+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7fa59c0479f0 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.750+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fa59c016b50 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.906+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa58c005870 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.907+0000 7fa5a2ffd640 1 -- 172.21.15.131:0/4103346190 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+318 (unknown 1187553405 0 999030115) 0x7fa59c016550 con 0x7fa5ac100370 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.909+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 >> v1:172.21.15.131:6800/1625784054 conn(0x7fa59803a600 legacy=0x7fa59803cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.909+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 >> v1:172.21.15.131:6789/0 conn(0x7fa5ac100370 legacy=0x7fa5ac10ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.909+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 shutdown_connections 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.909+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 >> 172.21.15.131:0/4103346190 conn(0x7fa5ac0fbb90 msgr2=0x7fa5ac0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.910+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 shutdown_connections 2024-09-10T18:33:34.519 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:33.910+0000 7fa5b0cab640 1 -- 172.21.15.131:0/4103346190 wait complete. 2024-09-10T18:33:34.520 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling cephadm module... 2024-09-10T18:33:35.476 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1141946658' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:33:35.477 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ignoring --setuser ceph since I am not root 2024-09-10T18:33:35.477 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ignoring --setgroup ceph since I am not root 2024-09-10T18:33:35.477 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:35.351+0000 7fcad58dbec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:33:35.477 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:35.405+0000 7fcad58dbec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:33:35.477 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:35.474+0000 7fcad58dbec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:33:35.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:35.598+0000 7fcad58dbec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:33:35.737 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:35.735+0000 7fcad58dbec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:33:35.820 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.959+0000 7f3420500640 1 Processor -- start 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.959+0000 7f3420500640 1 -- start start 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f3420500640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3418100e90 con 0x7f3418100370 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341e29c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3418100370 0x7f3418100780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44942/0 (socket says 172.21.15.131:44942) 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341e29c640 1 -- 172.21.15.131:0/4251864959 learned_addr learned my addr 172.21.15.131:0/4251864959 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3237760655 0 0) 0x7f3418100e90 con 0x7f3418100370 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34040034e0 con 0x7f3418100370 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 98142062 0 0) 0x7f34040034e0 con 0x7f3418100370 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3418105880 con 0x7f3418100370 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f340c001590 con 0x7f3418100370 2024-09-10T18:33:35.821 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f340c001cb0 con 0x7f3418100370 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.960+0000 7f341d29a640 1 -- 172.21.15.131:0/4251864959 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f340c005670 con 0x7f3418100370 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.961+0000 7f3420500640 1 -- 172.21.15.131:0/4251864959 >> v1:172.21.15.131:6789/0 conn(0x7f3418100370 legacy=0x7f3418100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.961+0000 7f3420500640 1 -- 172.21.15.131:0/4251864959 shutdown_connections 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.961+0000 7f3420500640 1 -- 172.21.15.131:0/4251864959 >> 172.21.15.131:0/4251864959 conn(0x7f34180fbb90 msgr2=0x7f34180fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.961+0000 7f3420500640 1 -- 172.21.15.131:0/4251864959 shutdown_connections 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.961+0000 7f3420500640 1 -- 172.21.15.131:0/4251864959 wait complete. 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.961+0000 7f3420500640 1 Processor -- start 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f3420500640 1 -- start start 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f3420500640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f341810f2f0 con 0x7f3418100370 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f341e29c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3418100370 0x7f341810ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44956/0 (socket says 172.21.15.131:44956) 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f341e29c640 1 -- 172.21.15.131:0/1141946658 learned_addr learned my addr 172.21.15.131:0/1141946658 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:35.822 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3318048344 0 0) 0x7f341810f2f0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34040053f0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 103381298 0 0) 0x7f34040053f0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f341810f2f0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f340c0047a0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 649287996 0 0) 0x7f341810f2f0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341810f480 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341810f750 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3418113230 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f340c004ba0 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.962+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f340c006090 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.963+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 4) v1 ==== 45446+0+0 (unknown 3805857646 0 0) 0x7f340c011380 con 0x7f3418100370 2024-09-10T18:33:35.823 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.963+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc005040 con 0x7f3418100370 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.963+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f340c047870 con 0x7f3418100370 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:34.968+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f340c012b20 con 0x7f3418100370 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.164+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f33fc0052f0 con 0x7f3418100370 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.217+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mgrmap(e 5) v1 ==== 45457+0+0 (unknown 2736308799 0 0) 0x7f340c046960 con 0x7f3418100370 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.218+0000 7f340affd640 1 -- 172.21.15.131:0/1141946658 <== mon.0 v1:172.21.15.131:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7f340c004d10 con 0x7f3418100370 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.221+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 >> v1:172.21.15.131:6800/1625784054 conn(0x7f340403a650 legacy=0x7f340403cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.221+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 >> v1:172.21.15.131:6789/0 conn(0x7f3418100370 legacy=0x7f341810ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.221+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 shutdown_connections 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.221+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 >> 172.21.15.131:0/1141946658 conn(0x7f34180fbb90 msgr2=0x7f34180fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.221+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 shutdown_connections 2024-09-10T18:33:35.824 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:35.221+0000 7f3420500640 1 -- 172.21.15.131:0/1141946658 wait complete. 2024-09-10T18:33:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1141946658' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:33:36.269 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:36 smithi131 ceph-mon[29369]: mgrmap e5: y(active, since 6s) 2024-09-10T18:33:36.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:35.998+0000 7fcad58dbec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:33:36.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:36 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:36.061+0000 7fcad58dbec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:33:36.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:36 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:36.229+0000 7fcad58dbec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:33:37.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:33:37.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.266+0000 7f54b9179640 1 Processor -- start 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.266+0000 7f54b9179640 1 -- start start 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b9179640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f54b4100f10 con 0x7f54b41003f0 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b2d9d640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f54b41003f0 0x7f54b4100800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44978/0 (socket says 172.21.15.131:44978) 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b2d9d640 1 -- 172.21.15.131:0/2727039548 learned_addr learned my addr 172.21.15.131:0/2727039548 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 141320802 0 0) 0x7f54b4100f10 con 0x7f54b41003f0 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54a00034e0 con 0x7f54b41003f0 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3954195124 0 0) 0x7f54a00034e0 con 0x7f54b41003f0 2024-09-10T18:33:37.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4105900 con 0x7f54b41003f0 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.267+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f54a4001590 con 0x7f54b41003f0 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f54a4001cb0 con 0x7f54b41003f0 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b259c640 1 -- 172.21.15.131:0/2727039548 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f54a4005670 con 0x7f54b41003f0 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b9179640 1 -- 172.21.15.131:0/2727039548 >> v1:172.21.15.131:6789/0 conn(0x7f54b41003f0 legacy=0x7f54b4100800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b9179640 1 -- 172.21.15.131:0/2727039548 shutdown_connections 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b9179640 1 -- 172.21.15.131:0/2727039548 >> 172.21.15.131:0/2727039548 conn(0x7f54b40fbb60 msgr2=0x7f54b40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b9179640 1 -- 172.21.15.131:0/2727039548 shutdown_connections 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.268+0000 7f54b9179640 1 -- 172.21.15.131:0/2727039548 wait complete. 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54b9179640 1 Processor -- start 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54b9179640 1 -- start start 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54b9179640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f54b4076120 con 0x7f54b41003f0 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54b2d9d640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f54b41003f0 0x7f54b4075a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:44994/0 (socket says 172.21.15.131:44994) 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54b2d9d640 1 -- 172.21.15.131:0/2181629778 learned_addr learned my addr 172.21.15.131:0/2181629778 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:37.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950325356 0 0) 0x7f54b4076120 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54a00053f0 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3668277832 0 0) 0x7f54a00053f0 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f54b4076120 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f54a40047a0 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1662853287 0 0) 0x7f54b4076120 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.269+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b4115d60 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.270+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54b4114db0 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.270+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f54b4115280 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.270+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f54a4004ba0 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.270+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f54a4006220 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.270+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 5) v1 ==== 45457+0+0 (unknown 2736308799 0 0) 0x7f54a4011520 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.271+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5494005040 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.271+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f54a40479c0 con 0x7f54b41003f0 2024-09-10T18:33:37.123 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.271+0000 7f54abfff640 1 -- 172.21.15.131:0/2181629778 >> v1:172.21.15.131:6800/1625784054 conn(0x7f54a003a640 legacy=0x7f54a003caf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/1625784054 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.275+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f54a4012da0 con 0x7f54b41003f0 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.472+0000 7f54abfff640 1 -- 172.21.15.131:0/2181629778 >> v1:172.21.15.131:6800/1625784054 conn(0x7f54a003a640 legacy=0x7f54a003caf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/1625784054 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.474+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f5494005b00 con 0x7f54b41003f0 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.474+0000 7f54ab7fe640 1 -- 172.21.15.131:0/2181629778 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7f54a40165b0 con 0x7f54b41003f0 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.476+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 >> v1:172.21.15.131:6800/1625784054 conn(0x7f54a003a640 legacy=0x7f54a003caf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.476+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 >> v1:172.21.15.131:6789/0 conn(0x7f54b41003f0 legacy=0x7f54b4075a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.476+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 shutdown_connections 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.476+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 >> 172.21.15.131:0/2181629778 conn(0x7f54b40fbb60 msgr2=0x7f54b4105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.476+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 shutdown_connections 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:36.476+0000 7f54b9179640 1 -- 172.21.15.131:0/2181629778 wait complete. 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-09-10T18:33:37.124 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 5... 2024-09-10T18:33:37.371 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2181629778' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:33:37.371 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:37 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:37.087+0000 7fcad58dbec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:33:37.371 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:37 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:37.156+0000 7fcad58dbec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:33:38.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:37 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:37.876+0000 7fcad58dbec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:33:38.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:37 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:37.986+0000 7fcad58dbec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:33:38.359 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.106+0000 7fcad58dbec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:33:38.359 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.165+0000 7fcad58dbec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:33:38.359 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.290+0000 7fcad58dbec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:33:38.359 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.357+0000 7fcad58dbec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:33:38.613 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.414+0000 7fcad58dbec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:33:38.613 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.547+0000 7fcad58dbec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:33:38.613 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.610+0000 7fcad58dbec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:33:38.875 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.675+0000 7fcad58dbec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:33:38.875 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.755+0000 7fcad58dbec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:33:38.875 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.814+0000 7fcad58dbec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:33:38.876 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:38.873+0000 7fcad58dbec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:33:39.678 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:39.622+0000 7fcad58dbec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: Active manager daemon y restarted 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: Activating manager daemon y 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: mgrmap e6: y(active, starting, since 0.00313867s) 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:33:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: Manager daemon y is now available 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:33:40.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:39 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:33:40.935 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:40 smithi131 ceph-mon[29369]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:33:40.935 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:40 smithi131 ceph-mon[29369]: mgrmap e7: y(active, since 1.00514s) 2024-09-10T18:33:41.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:41.037+0000 7fca87cc9640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:33:41] ENGINE Error in HTTPServer.serve 2024-09-10T18:33:41.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: Traceback (most recent call last): 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._connections.run(self.expiration_interval) 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._run(expiration_interval) 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: s = self.context.wrap_socket( 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: return self.sslsocket_class._create( 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:33:41.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self.do_handshake() 2024-09-10T18:33:41.329 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:33:41.329 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._sslobj.do_handshake() 2024-09-10T18:33:41.329 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:33:41.329 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:41 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.590+0000 7f504c852640 1 Processor -- start 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.590+0000 7f504c852640 1 -- start start 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504c852640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5048100f20 con 0x7f5048100400 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504659c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5048100400 0x7f5048100810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45008/0 (socket says 172.21.15.131:45008) 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504659c640 1 -- 172.21.15.131:0/935857076 learned_addr learned my addr 172.21.15.131:0/935857076 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:41.436 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055718723 0 0) 0x7f5048100f20 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50300034e0 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082525806 0 0) 0x7f50300034e0 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048105910 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5038001590 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f5038001cb0 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.591+0000 7f504559a640 1 -- 172.21.15.131:0/935857076 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5038005670 con 0x7f5048100400 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.592+0000 7f504c852640 1 -- 172.21.15.131:0/935857076 >> v1:172.21.15.131:6789/0 conn(0x7f5048100400 legacy=0x7f5048100810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.592+0000 7f504c852640 1 -- 172.21.15.131:0/935857076 shutdown_connections 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.592+0000 7f504c852640 1 -- 172.21.15.131:0/935857076 >> 172.21.15.131:0/935857076 conn(0x7f50480fbb30 msgr2=0x7f50480fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.592+0000 7f504c852640 1 -- 172.21.15.131:0/935857076 shutdown_connections 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.592+0000 7f504c852640 1 -- 172.21.15.131:0/935857076 wait complete. 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f504c852640 1 Processor -- start 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f504c852640 1 -- start start 2024-09-10T18:33:41.437 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f504c852640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f504810f330 con 0x7f5048100400 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f504659c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5048100400 0x7f504810ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45010/0 (socket says 172.21.15.131:45010) 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f504659c640 1 -- 172.21.15.131:0/1993446917 learned_addr learned my addr 172.21.15.131:0/1993446917 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3451101827 0 0) 0x7f504810f330 con 0x7f5048100400 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50300053f0 con 0x7f5048100400 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2744859742 0 0) 0x7f50300053f0 con 0x7f5048100400 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f504810f330 con 0x7f5048100400 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.593+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5038001060 con 0x7f5048100400 2024-09-10T18:33:41.438 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.594+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1997744887 0 0) 0x7f504810f330 con 0x7f5048100400 2024-09-10T18:33:41.439 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.594+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504810f4c0 con 0x7f5048100400 2024-09-10T18:33:41.439 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.594+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504810f790 con 0x7f5048100400 2024-09-10T18:33:41.439 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.594+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5048113270 con 0x7f5048100400 2024-09-10T18:33:41.439 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.594+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f5038004ba0 con 0x7f5048100400 2024-09-10T18:33:41.439 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.594+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f50380060d0 con 0x7f5048100400 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.595+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 5) v1 ==== 45457+0+0 (unknown 2736308799 0 0) 0x7f50380113d0 con 0x7f5048100400 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.595+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6800/1625784054 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f503003e310 con 0x7f503003a650 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.595+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (unknown 2867510069 0 0) 0x7f50380478e0 con 0x7f5048100400 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.595+0000 7f5045d9b640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6800/1625784054 conn(0x7f503003a650 legacy=0x7f503003cb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/1625784054 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:37.796+0000 7f5045d9b640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6800/1625784054 conn(0x7f503003a650 legacy=0x7f503003cb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/1625784054 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:38.197+0000 7f5045d9b640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6800/1625784054 conn(0x7f503003a650 legacy=0x7f503003cb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/1625784054 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:38.998+0000 7f5045d9b640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6800/1625784054 conn(0x7f503003a650 legacy=0x7f503003cb00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/1625784054 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:39.625+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mgrmap(e 6) v1 ==== 45259+0+0 (unknown 2342565245 0 0) 0x7f5038046a60 con 0x7f5048100400 2024-09-10T18:33:41.440 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:39.625+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6800/1625784054 conn(0x7f503003a650 legacy=0x7f503003cb00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.626+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mgrmap(e 7) v1 ==== 45351+0+0 (unknown 1205712984 0 0) 0x7f5038012830 con 0x7f5048100400 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.626+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6800/2393239631 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f503003e310 con 0x7f503003efd0 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.627+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (unknown 0 0 4193150809) 0x7f503003e310 con 0x7f503003efd0 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.632+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 --> v1:172.21.15.131:6800/2393239631 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f5028002560 con 0x7f503003efd0 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.632+0000 7f5036ffd640 1 -- 172.21.15.131:0/1993446917 <== mgr.14118 v1:172.21.15.131:6800/2393239631 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7f5028002560 con 0x7f503003efd0 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.633+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6800/2393239631 conn(0x7f503003efd0 legacy=0x7f50300413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.633+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 >> v1:172.21.15.131:6789/0 conn(0x7f5048100400 legacy=0x7f504810ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.633+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 shutdown_connections 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.633+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 >> 172.21.15.131:0/1993446917 conn(0x7f50480fbb30 msgr2=0x7f5048105ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.633+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 shutdown_connections 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:40.633+0000 7f504c852640 1 -- 172.21.15.131:0/1993446917 wait complete. 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 5 is available 2024-09-10T18:33:41.441 INFO:teuthology.orchestra.run.smithi131.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:33:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: from='client.14122 v1:172.21.15.131:0/1993446917' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: from='client.14122 v1:172.21.15.131:0/1993446917' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: [10/Sep/2024:18:33:40] ENGINE Bus STARTING 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: [10/Sep/2024:18:33:41] ENGINE Serving on https://172.21.15.131:7150 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: [10/Sep/2024:18:33:41] ENGINE Bus STARTED 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: [10/Sep/2024:18:33:41] ENGINE Error in HTTPServer.serve 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:33:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:33:42.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:42.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:42 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:42.641 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c9e94b640 1 Processor -- start 2024-09-10T18:33:42.641 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c9e94b640 1 -- start start 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c9e94b640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7c98100e40 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c97fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7c98100320 0x7f7c98100730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51946/0 (socket says 172.21.15.131:51946) 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c97fff640 1 -- 172.21.15.131:0/2490893805 learned_addr learned my addr 172.21.15.131:0/2490893805 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3994127609 0 0) 0x7f7c98100e40 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c800034e0 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2757343085 0 0) 0x7f7c800034e0 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98105870 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.880+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7c88001590 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f7c88001cb0 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c96ffd640 1 -- 172.21.15.131:0/2490893805 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7c88005670 con 0x7f7c98100320 2024-09-10T18:33:42.642 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c9e94b640 1 -- 172.21.15.131:0/2490893805 >> v1:172.21.15.131:6789/0 conn(0x7f7c98100320 legacy=0x7f7c98100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c9e94b640 1 -- 172.21.15.131:0/2490893805 shutdown_connections 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c9e94b640 1 -- 172.21.15.131:0/2490893805 >> 172.21.15.131:0/2490893805 conn(0x7f7c980fbba0 msgr2=0x7f7c980fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c9e94b640 1 -- 172.21.15.131:0/2490893805 shutdown_connections 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.881+0000 7f7c9e94b640 1 -- 172.21.15.131:0/2490893805 wait complete. 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c9e94b640 1 Processor -- start 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c9e94b640 1 -- start start 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c9e94b640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7c9810f350 con 0x7f7c98100320 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c97fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7c98100320 0x7f7c9810ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51954/0 (socket says 172.21.15.131:51954) 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c97fff640 1 -- 172.21.15.131:0/3895151585 learned_addr learned my addr 172.21.15.131:0/3895151585 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029725351 0 0) 0x7f7c9810f350 con 0x7f7c98100320 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c800053f0 con 0x7f7c98100320 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 809578764 0 0) 0x7f7c800053f0 con 0x7f7c98100320 2024-09-10T18:33:42.643 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7c9810f350 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.882+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7c880047a0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2785005202 0 0) 0x7f7c9810f350 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9810f4e0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9810f7b0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c98113290 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f7c88004ba0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7c880061c0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.883+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 7) v1 ==== 45351+0+0 (unknown 1205712984 0 0) 0x7f7c88011470 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.884+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f7c880476e0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.884+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c84005040 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:41.888+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7c88013110 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.040+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7f7c88012bc0 con 0x7f7c98100320 2024-09-10T18:33:42.644 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.055+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c84000bc0 con 0x7f7c8003a580 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.064+0000 7f7c94ff9640 1 -- 172.21.15.131:0/3895151585 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f7c84000bc0 con 0x7f7c8003a580 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.066+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 >> v1:172.21.15.131:6800/2393239631 conn(0x7f7c8003a580 legacy=0x7f7c8003ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.066+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 >> v1:172.21.15.131:6789/0 conn(0x7f7c98100320 legacy=0x7f7c9810ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.066+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 shutdown_connections 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.066+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 >> 172.21.15.131:0/3895151585 conn(0x7f7c980fbba0 msgr2=0x7f7c98105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.066+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 shutdown_connections 2024-09-10T18:33:42.645 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:42.066+0000 7f7c9e94b640 1 -- 172.21.15.131:0/3895151585 wait complete. 2024-09-10T18:33:43.250 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:43 smithi131 ceph-mon[29369]: mgrmap e8: y(active, since 2s) 2024-09-10T18:33:43.250 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:43 smithi131 ceph-mon[29369]: from='client.14130 v1:172.21.15.131:0/3895151585' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:43.250 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:43 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:43.250 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:43 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:43.840 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:33:43.840 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.071+0000 7f875111c640 1 Processor -- start 2024-09-10T18:33:43.840 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.071+0000 7f875111c640 1 -- start start 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.071+0000 7f875111c640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f874c076100 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f874ad9d640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f874c075cf0 0x7f874c074270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51956/0 (socket says 172.21.15.131:51956) 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f874ad9d640 1 -- 172.21.15.131:0/623052200 learned_addr learned my addr 172.21.15.131:0/623052200 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2907644662 0 0) 0x7f874c076100 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87340034e0 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1214198811 0 0) 0x7f87340034e0 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c074980 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f873c001590 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f873c001be0 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.072+0000 7f8749d9b640 1 -- 172.21.15.131:0/623052200 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f873c0055b0 con 0x7f874c075cf0 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- 172.21.15.131:0/623052200 >> v1:172.21.15.131:6789/0 conn(0x7f874c075cf0 legacy=0x7f874c074270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:43.841 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- 172.21.15.131:0/623052200 shutdown_connections 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- 172.21.15.131:0/623052200 >> 172.21.15.131:0/623052200 conn(0x7f874c0fbb10 msgr2=0x7f874c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- 172.21.15.131:0/623052200 shutdown_connections 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- 172.21.15.131:0/623052200 wait complete. 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 Processor -- start 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- start start 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.073+0000 7f875111c640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f874c111610 con 0x7f874c075cf0 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f874ad9d640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f874c075cf0 0x7f874c110f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51964/0 (socket says 172.21.15.131:51964) 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f874ad9d640 1 -- 172.21.15.131:0/1771259774 learned_addr learned my addr 172.21.15.131:0/1771259774 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659739349 0 0) 0x7f874c111610 con 0x7f874c075cf0 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8734005440 con 0x7f874c075cf0 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2522895647 0 0) 0x7f8734005440 con 0x7f874c075cf0 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f874c111610 con 0x7f874c075cf0 2024-09-10T18:33:43.842 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f873c001d50 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3650836561 0 0) 0x7f874c111610 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c1117a0 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c111a70 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c115550 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f873c0017e0 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.074+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f873c006010 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.075+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7f873c0113d0 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.075+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8738005040 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.075+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f873c047ae0 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.080+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f873c012de0 con 0x7f874c075cf0 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.236+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f8738000bc0 con 0x7f873403a650 2024-09-10T18:33:43.843 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.237+0000 7f87337fe640 1 -- 172.21.15.131:0/1771259774 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7f8738000bc0 con 0x7f873403a650 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.239+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 >> v1:172.21.15.131:6800/2393239631 conn(0x7f873403a650 legacy=0x7f873403cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.239+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 >> v1:172.21.15.131:6789/0 conn(0x7f874c075cf0 legacy=0x7f874c110f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.239+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 shutdown_connections 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.239+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 >> 172.21.15.131:0/1771259774 conn(0x7f874c0fbb10 msgr2=0x7f874c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.239+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 shutdown_connections 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:43.239+0000 7f875111c640 1 -- 172.21.15.131:0/1771259774 wait complete. 2024-09-10T18:33:43.844 INFO:teuthology.orchestra.run.smithi131.stdout:Generating ssh key... 2024-09-10T18:33:44.186 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-mon[29369]: from='client.14132 v1:172.21.15.131:0/1771259774' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:44.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: Generating public/private rsa key pair. 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: Your identification has been saved in /tmp/tmpcshohv18/key 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: Your public key has been saved in /tmp/tmpcshohv18/key.pub 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: The key fingerprint is: 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: SHA256:hfi1pgx2vVSfhCqeJ04kH/9vbrDgXve47o5UjplBFbA ceph-0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: The key's randomart image is: 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: +---[RSA 3072]----+ 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | ..o. | 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | . . + | 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | . . o E . | 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | . + = o . | 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | + S * . + | 2024-09-10T18:33:44.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | . O O...B | 2024-09-10T18:33:44.829 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | O.+.*oo | 2024-09-10T18:33:44.829 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | o o.+.ooo | 2024-09-10T18:33:44.829 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: | ... oOO..| 2024-09-10T18:33:44.829 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:44 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: +----[SHA256]-----+ 2024-09-10T18:33:45.114 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.259+0000 7fdccc179640 1 Processor -- start 2024-09-10T18:33:45.114 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.259+0000 7fdccc179640 1 -- start start 2024-09-10T18:33:45.114 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdccc179640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdcc40ffd70 con 0x7fdcc40ff250 2024-09-10T18:33:45.115 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc9f15640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdcc40ff250 0x7fdcc40ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51976/0 (socket says 172.21.15.131:51976) 2024-09-10T18:33:45.115 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc9f15640 1 -- 172.21.15.131:0/1924294861 learned_addr learned my addr 172.21.15.131:0/1924294861 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:45.115 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024941669 0 0) 0x7fdcc40ffd70 con 0x7fdcc40ff250 2024-09-10T18:33:45.115 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb00034e0 con 0x7fdcc40ff250 2024-09-10T18:33:45.115 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261161556 0 0) 0x7fdcb00034e0 con 0x7fdcc40ff250 2024-09-10T18:33:45.115 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4104710 con 0x7fdcc40ff250 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fdcc0001590 con 0x7fdcc40ff250 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fdcc0001cb0 con 0x7fdcc40ff250 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.260+0000 7fdcc8f13640 1 -- 172.21.15.131:0/1924294861 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fdcc0005670 con 0x7fdcc40ff250 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- 172.21.15.131:0/1924294861 >> v1:172.21.15.131:6789/0 conn(0x7fdcc40ff250 legacy=0x7fdcc40ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- 172.21.15.131:0/1924294861 shutdown_connections 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- 172.21.15.131:0/1924294861 >> 172.21.15.131:0/1924294861 conn(0x7fdcc40fa9d0 msgr2=0x7fdcc40fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:45.116 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- 172.21.15.131:0/1924294861 shutdown_connections 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- 172.21.15.131:0/1924294861 wait complete. 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 Processor -- start 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- start start 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdccc179640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdcc4112a40 con 0x7fdcc40ff250 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdcc9f15640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdcc40ff250 0x7fdcc4073910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51992/0 (socket says 172.21.15.131:51992) 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.261+0000 7fdcc9f15640 1 -- 172.21.15.131:0/150099678 learned_addr learned my addr 172.21.15.131:0/150099678 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344995309 0 0) 0x7fdcc4112a40 con 0x7fdcc40ff250 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb00053f0 con 0x7fdcc40ff250 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2598611674 0 0) 0x7fdcb00053f0 con 0x7fdcc40ff250 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdcc4112a40 con 0x7fdcc40ff250 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fdcc00047a0 con 0x7fdcc40ff250 2024-09-10T18:33:45.117 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 434508232 0 0) 0x7fdcc4112a40 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4113be0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc4112bd0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc41130a0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fdcc00049e0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.262+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fdcc0005ff0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.263+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7fdcc00113b0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.263+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8005040 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.263+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7fdcc0047ac0 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.267+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdcc0049e60 con 0x7fdcc40ff250 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.421+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fdca8000bc0 con 0x7fdcb003a650 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.520+0000 7fdcba7fc640 1 -- 172.21.15.131:0/150099678 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7fdca8000bc0 con 0x7fdcb003a650 2024-09-10T18:33:45.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.522+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 >> v1:172.21.15.131:6800/2393239631 conn(0x7fdcb003a650 legacy=0x7fdcb003cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.522+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 >> v1:172.21.15.131:6789/0 conn(0x7fdcc40ff250 legacy=0x7fdcc4073910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:45.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.522+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 shutdown_connections 2024-09-10T18:33:45.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.522+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 >> 172.21.15.131:0/150099678 conn(0x7fdcc40fa9d0 msgr2=0x7fdcc4104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:45.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.522+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 shutdown_connections 2024-09-10T18:33:45.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:44.522+0000 7fdccc179640 1 -- 172.21.15.131:0/150099678 wait complete. 2024-09-10T18:33:45.727 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:45 smithi131 ceph-mon[29369]: from='client.14134 v1:172.21.15.131:0/150099678' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:45.727 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:45 smithi131 ceph-mon[29369]: Generating ssh key... 2024-09-10T18:33:45.727 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:45 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:45.727 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:45 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:46.328 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIKlXr8lmOQ8xdiCFBGPnUqPnHrCElXrtJ2l3sDjoM1lewhJ7uYeU/5qSIrRWTlFZbZFGXIsUH/cTo5iVkgii8ZJ/KMKAP8aRjtb9xEQ9kVqwi7sb+IPHD+bm0+CQXEMSnJ5ooqFpIcfBB17K+0rCSzpjykpAvaqz8+hND4LHqIU7BHzMwrVchmrHdunT7i8TfYWSdguQLwVYXI93SEQJOGO1EsK+Mm9ntASz+lcQE0OcXYrhN6o1eiJ8eL34+NeEyj7f1eiBmzEfLDqNr10rpDKBTP3BEu6kw3qyNUYCEvbm9E2e4DxWdiN1vbi4rshQLQuIqSICYw+SKwQaMjDxEMSxZINQvdN4ua5ciDr+wJ6U+90pFnfg2NuANrgMOazMmPxCQ8KLfp48sjTfOuFvPPCTT+etvi0Wa6rVfJomBYH/mA1Km+y5JgDZ72lTI7APbnwqIyVZi2oUftSZB+0zOUahXgITOlDJhCybrLhnjVEEMJia2oGvPq2PIytE3/8E= ceph-0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:33:46.328 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5da1d95640 1 Processor -- start 2024-09-10T18:33:46.328 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5da1d95640 1 -- start start 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5da1d95640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d9c100e40 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5d9b7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5d9c100320 0x7f5d9c100730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51996/0 (socket says 172.21.15.131:51996) 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5d9b7fe640 1 -- 172.21.15.131:0/3030618438 learned_addr learned my addr 172.21.15.131:0/3030618438 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328351820 0 0) 0x7f5d9c100e40 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.558+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d840034e0 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2628889748 0 0) 0x7f5d840034e0 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c105870 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5d8c001590 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f5d8c001cb0 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5d9a7fc640 1 -- 172.21.15.131:0/3030618438 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5d8c005670 con 0x7f5d9c100320 2024-09-10T18:33:46.329 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5da1d95640 1 -- 172.21.15.131:0/3030618438 >> v1:172.21.15.131:6789/0 conn(0x7f5d9c100320 legacy=0x7f5d9c100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5da1d95640 1 -- 172.21.15.131:0/3030618438 shutdown_connections 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5da1d95640 1 -- 172.21.15.131:0/3030618438 >> 172.21.15.131:0/3030618438 conn(0x7f5d9c0fbba0 msgr2=0x7f5d9c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5da1d95640 1 -- 172.21.15.131:0/3030618438 shutdown_connections 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.559+0000 7f5da1d95640 1 -- 172.21.15.131:0/3030618438 wait complete. 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5da1d95640 1 Processor -- start 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5da1d95640 1 -- start start 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5da1d95640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d9c10f350 con 0x7f5d9c100320 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5d9b7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f5d9c100320 0x7f5d9c10ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51998/0 (socket says 172.21.15.131:51998) 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5d9b7fe640 1 -- 172.21.15.131:0/1587302044 learned_addr learned my addr 172.21.15.131:0/1587302044 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905989613 0 0) 0x7f5d9c10f350 con 0x7f5d9c100320 2024-09-10T18:33:46.330 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.560+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d840053f0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3360023451 0 0) 0x7f5d840053f0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d9c10f350 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5d8c0047a0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1650381667 0 0) 0x7f5d9c10f350 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c10f4e0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c10f7b0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c113290 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7f5d8c004ba0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.561+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f5d8c0061c0 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.562+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7f5d8c006490 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.562+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d88005040 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.562+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f5d8c047a60 con 0x7f5d9c100320 2024-09-10T18:33:46.331 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.566+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f5d8c007280 con 0x7f5d9c100320 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.725+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d88000bc0 con 0x7f5d8403a650 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.726+0000 7f5d83fff640 1 -- 172.21.15.131:0/1587302044 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 872975028) 0x7f5d88000bc0 con 0x7f5d8403a650 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.728+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 >> v1:172.21.15.131:6800/2393239631 conn(0x7f5d8403a650 legacy=0x7f5d8403cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.728+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 >> v1:172.21.15.131:6789/0 conn(0x7f5d9c100320 legacy=0x7f5d9c10ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.728+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 shutdown_connections 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.728+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 >> 172.21.15.131:0/1587302044 conn(0x7f5d9c0fbba0 msgr2=0x7f5d9c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.728+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 shutdown_connections 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:45.728+0000 7f5da1d95640 1 -- 172.21.15.131:0/1587302044 wait complete. 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:33:46.332 INFO:teuthology.orchestra.run.smithi131.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:33:46.333 INFO:teuthology.orchestra.run.smithi131.stdout:Adding host smithi131... 2024-09-10T18:33:46.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:46 smithi131 ceph-mon[29369]: from='client.14136 v1:172.21.15.131:0/1587302044' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:47.690 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:47 smithi131 ceph-mon[29369]: from='client.14138 v1:172.21.15.131:0/4017165063' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:48.657 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:48 smithi131 ceph-mon[29369]: Deploying cephadm binary to smithi131 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Added host 'smithi131' with addr '172.21.15.131' 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.776+0000 7fe875c41640 1 Processor -- start 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.776+0000 7fe875c41640 1 -- start start 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.776+0000 7fe875c41640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe8701046d0 con 0x7fe870074500 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.776+0000 7fe86f7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe870074500 0x7fe870102fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52000/0 (socket says 172.21.15.131:52000) 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.776+0000 7fe86f7fe640 1 -- 172.21.15.131:0/1808587821 learned_addr learned my addr 172.21.15.131:0/1808587821 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988493923 0 0) 0x7fe8701046d0 con 0x7fe870074500 2024-09-10T18:33:50.220 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8580034e0 con 0x7fe870074500 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265605486 0 0) 0x7fe8580034e0 con 0x7fe870074500 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe870105870 con 0x7fe870074500 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe860001590 con 0x7fe870074500 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fe860001cb0 con 0x7fe870074500 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe86e7fc640 1 -- 172.21.15.131:0/1808587821 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe860005600 con 0x7fe870074500 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe875c41640 1 -- 172.21.15.131:0/1808587821 >> v1:172.21.15.131:6789/0 conn(0x7fe870074500 legacy=0x7fe870102fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe875c41640 1 -- 172.21.15.131:0/1808587821 shutdown_connections 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.777+0000 7fe875c41640 1 -- 172.21.15.131:0/1808587821 >> 172.21.15.131:0/1808587821 conn(0x7fe8700faaa0 msgr2=0x7fe8700fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.778+0000 7fe875c41640 1 -- 172.21.15.131:0/1808587821 shutdown_connections 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.778+0000 7fe875c41640 1 -- 172.21.15.131:0/1808587821 wait complete. 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.778+0000 7fe875c41640 1 Processor -- start 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.778+0000 7fe875c41640 1 -- start start 2024-09-10T18:33:50.221 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.778+0000 7fe875c41640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe870111510 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe86f7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe870074500 0x7fe870110e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:52006/0 (socket says 172.21.15.131:52006) 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe86f7fe640 1 -- 172.21.15.131:0/4017165063 learned_addr learned my addr 172.21.15.131:0/4017165063 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993347233 0 0) 0x7fe870111510 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8580053f0 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4262404511 0 0) 0x7fe8580053f0 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe870111510 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe8600047a0 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4166956804 0 0) 0x7fe870111510 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8701116a0 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe870111950 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8701153c0 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fe860004b30 con 0x7fe870074500 2024-09-10T18:33:50.222 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.779+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fe8600060b0 con 0x7fe870074500 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.780+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7fe860006360 con 0x7fe870074500 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.780+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe85c005040 con 0x7fe870074500 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.780+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7fe860047a30 con 0x7fe870074500 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.785+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fe860007150 con 0x7fe870074500 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:46.943+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi131", "addr": "172.21.15.131", "target": ["mon-mgr", ""]}) v1 -- 0x7fe85c000bc0 con 0x7fe85803a6f0 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.637+0000 7fe857fff640 1 -- 172.21.15.131:0/4017165063 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 2335836962) 0x7fe85c000bc0 con 0x7fe85803a6f0 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.640+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 >> v1:172.21.15.131:6800/2393239631 conn(0x7fe85803a6f0 legacy=0x7fe85803cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.640+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 >> v1:172.21.15.131:6789/0 conn(0x7fe870074500 legacy=0x7fe870110e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.640+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 shutdown_connections 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.640+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 >> 172.21.15.131:0/4017165063 conn(0x7fe8700faaa0 msgr2=0x7fe8700fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:50.223 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.640+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 shutdown_connections 2024-09-10T18:33:50.224 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:49.640+0000 7fe875c41640 1 -- 172.21.15.131:0/4017165063 wait complete. 2024-09-10T18:33:50.224 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mon service... 2024-09-10T18:33:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:50 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:50 smithi131 ceph-mon[29369]: Added host smithi131 2024-09-10T18:33:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:50 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:33:51.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:33:51.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.670+0000 7fd327906640 1 Processor -- start 2024-09-10T18:33:51.803 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.670+0000 7fd327906640 1 -- start start 2024-09-10T18:33:51.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.670+0000 7fd327906640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd320100e90 con 0x7fd320100370 2024-09-10T18:33:51.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.670+0000 7fd3256a2640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd320100370 0x7fd320100780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41206/0 (socket says 172.21.15.131:41206) 2024-09-10T18:33:51.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.670+0000 7fd3256a2640 1 -- 172.21.15.131:0/719111396 learned_addr learned my addr 172.21.15.131:0/719111396 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:51.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810049446 0 0) 0x7fd320100e90 con 0x7fd320100370 2024-09-10T18:33:51.804 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30c0034e0 con 0x7fd320100370 2024-09-10T18:33:51.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843287386 0 0) 0x7fd30c0034e0 con 0x7fd320100370 2024-09-10T18:33:51.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320105880 con 0x7fd320100370 2024-09-10T18:33:51.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd31c001590 con 0x7fd320100370 2024-09-10T18:33:51.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fd31c001cb0 con 0x7fd320100370 2024-09-10T18:33:51.805 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd317fff640 1 -- 172.21.15.131:0/719111396 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd31c005670 con 0x7fd320100370 2024-09-10T18:33:51.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd327906640 1 -- 172.21.15.131:0/719111396 >> v1:172.21.15.131:6789/0 conn(0x7fd320100370 legacy=0x7fd320100780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:51.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.671+0000 7fd327906640 1 -- 172.21.15.131:0/719111396 shutdown_connections 2024-09-10T18:33:51.806 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd327906640 1 -- 172.21.15.131:0/719111396 >> 172.21.15.131:0/719111396 conn(0x7fd3200fbb90 msgr2=0x7fd3200fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:51.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd327906640 1 -- 172.21.15.131:0/719111396 shutdown_connections 2024-09-10T18:33:51.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd327906640 1 -- 172.21.15.131:0/719111396 wait complete. 2024-09-10T18:33:51.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd327906640 1 Processor -- start 2024-09-10T18:33:51.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd327906640 1 -- start start 2024-09-10T18:33:51.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd327906640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd320076a00 con 0x7fd320100370 2024-09-10T18:33:51.807 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd3256a2640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fd320100370 0x7fd3200762f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41214/0 (socket says 172.21.15.131:41214) 2024-09-10T18:33:51.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.672+0000 7fd3256a2640 1 -- 172.21.15.131:0/2723550936 learned_addr learned my addr 172.21.15.131:0/2723550936 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:51.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314651170 0 0) 0x7fd320076a00 con 0x7fd320100370 2024-09-10T18:33:51.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30c0053f0 con 0x7fd320100370 2024-09-10T18:33:51.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2938791485 0 0) 0x7fd30c0053f0 con 0x7fd320100370 2024-09-10T18:33:51.808 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd320076a00 con 0x7fd320100370 2024-09-10T18:33:51.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd31c0047a0 con 0x7fd320100370 2024-09-10T18:33:51.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3914235946 0 0) 0x7fd320076a00 con 0x7fd320100370 2024-09-10T18:33:51.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320115d10 con 0x7fd320100370 2024-09-10T18:33:51.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd320114d60 con 0x7fd320100370 2024-09-10T18:33:51.809 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 963+0+0 (unknown 2256456929 0 0) 0x7fd31c004ba0 con 0x7fd320100370 2024-09-10T18:33:51.810 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fd31c006200 con 0x7fd320100370 2024-09-10T18:33:51.810 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.673+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd320115230 con 0x7fd320100370 2024-09-10T18:33:51.810 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.674+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7fd31c011650 con 0x7fd320100370 2024-09-10T18:33:51.810 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.674+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7fd31c047af0 con 0x7fd320100370 2024-09-10T18:33:51.810 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.674+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd320105390 con 0x7fd320100370 2024-09-10T18:33:51.811 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.679+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fd31c013250 con 0x7fd320100370 2024-09-10T18:33:51.811 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.843+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fd320115b80 con 0x7fd30c03a640 2024-09-10T18:33:51.811 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.850+0000 7fd315ffb640 1 -- 172.21.15.131:0/2723550936 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7fd320115b80 con 0x7fd30c03a640 2024-09-10T18:33:51.811 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.852+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 >> v1:172.21.15.131:6800/2393239631 conn(0x7fd30c03a640 legacy=0x7fd30c03caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:51.811 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.852+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 >> v1:172.21.15.131:6789/0 conn(0x7fd320100370 legacy=0x7fd3200762f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:51.812 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.852+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 shutdown_connections 2024-09-10T18:33:51.812 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.852+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 >> 172.21.15.131:0/2723550936 conn(0x7fd3200fbb90 msgr2=0x7fd3200fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:51.812 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.852+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 shutdown_connections 2024-09-10T18:33:51.812 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:50.852+0000 7fd327906640 1 -- 172.21.15.131:0/2723550936 wait complete. 2024-09-10T18:33:51.812 INFO:teuthology.orchestra.run.smithi131.stdout:Deploying unmanaged mgr service... 2024-09-10T18:33:52.046 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:51 smithi131 ceph-mon[29369]: from='client.14140 v1:172.21.15.131:0/2723550936' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:52.046 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:51 smithi131 ceph-mon[29369]: Saving service mon spec with placement count:5 2024-09-10T18:33:52.046 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:51 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.001+0000 7f7fac15f640 1 Processor -- start 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.001+0000 7f7fac15f640 1 -- start start 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fac15f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7fa41046e0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa9efb640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7fa4073bf0 0x7f7fa4102fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41224/0 (socket says 172.21.15.131:41224) 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa9efb640 1 -- 172.21.15.131:0/4226302398 learned_addr learned my addr 172.21.15.131:0/4226302398 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296042210 0 0) 0x7f7fa41046e0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f900034e0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1858806989 0 0) 0x7f7f900034e0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa4105880 con 0x7f7fa4073bf0 2024-09-10T18:33:53.869 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7f98001590 con 0x7f7fa4073bf0 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.002+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f7f98001cd0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.003+0000 7f7fa8ef9640 1 -- 172.21.15.131:0/4226302398 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7f980056c0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.003+0000 7f7fac15f640 1 -- 172.21.15.131:0/4226302398 >> v1:172.21.15.131:6789/0 conn(0x7f7fa4073bf0 legacy=0x7f7fa4102fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.003+0000 7f7fac15f640 1 -- 172.21.15.131:0/4226302398 shutdown_connections 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.003+0000 7f7fac15f640 1 -- 172.21.15.131:0/4226302398 >> 172.21.15.131:0/4226302398 conn(0x7f7fa40fa9d0 msgr2=0x7f7fa40fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.003+0000 7f7fac15f640 1 -- 172.21.15.131:0/4226302398 shutdown_connections 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.003+0000 7f7fac15f640 1 -- 172.21.15.131:0/4226302398 wait complete. 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7fac15f640 1 Processor -- start 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7fac15f640 1 -- start start 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7fac15f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7fa410f340 con 0x7f7fa4073bf0 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7fa9efb640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7fa4073bf0 0x7f7fa410ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41230/0 (socket says 172.21.15.131:41230) 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7fa9efb640 1 -- 172.21.15.131:0/2291725155 learned_addr learned my addr 172.21.15.131:0/2291725155 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1750239169 0 0) 0x7f7fa410f340 con 0x7f7fa4073bf0 2024-09-10T18:33:53.870 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f900053f0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2172795377 0 0) 0x7f7f900053f0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7fa410f340 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.004+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7f98001430 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.005+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4071754199 0 0) 0x7f7fa410f340 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.005+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa410f4d0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.005+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa410f7a0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.005+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa4113280 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.005+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f7f98001b30 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.005+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7f98006000 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.006+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7f7f980113a0 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.006+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f88005040 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.006+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f7f98048820 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.010+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7f98016c00 con 0x7f7fa4073bf0 2024-09-10T18:33:53.871 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.203+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 --> v1:172.21.15.131:6800/2393239631 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7f88000bc0 con 0x7f7f9003a620 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.212+0000 7f7f967fc640 1 -- 172.21.15.131:0/2291725155 <== mgr.14118 v1:172.21.15.131:6800/2393239631 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f7f88000bc0 con 0x7f7f9003a620 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.214+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 >> v1:172.21.15.131:6800/2393239631 conn(0x7f7f9003a620 legacy=0x7f7f9003cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.214+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 >> v1:172.21.15.131:6789/0 conn(0x7f7fa4073bf0 legacy=0x7f7fa410ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.214+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 shutdown_connections 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.214+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 >> 172.21.15.131:0/2291725155 conn(0x7f7fa40fa9d0 msgr2=0x7f7fa41028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.215+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 shutdown_connections 2024-09-10T18:33:53.872 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:53.215+0000 7f7fac15f640 1 -- 172.21.15.131:0/2291725155 wait complete. 2024-09-10T18:33:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:53 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:53 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:53 smithi131 ceph-mon[29369]: from='client.14142 v1:172.21.15.131:0/2291725155' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:33:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:53 smithi131 ceph-mon[29369]: Saving service mgr spec with placement count:2 2024-09-10T18:33:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:53 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:55.118 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.358+0000 7f408967f640 1 Processor -- start 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.358+0000 7f408967f640 1 -- start start 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.358+0000 7f408967f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4084104670 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.358+0000 7f4082ffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4084074560 0x7f4084102f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41234/0 (socket says 172.21.15.131:41234) 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.358+0000 7f4082ffd640 1 -- 172.21.15.131:0/3400233301 learned_addr learned my addr 172.21.15.131:0/3400233301 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161809493 0 0) 0x7f4084104670 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40700034e0 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2828920892 0 0) 0x7f40700034e0 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4084105810 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4074001590 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f4074001cd0 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f40827fc640 1 -- 172.21.15.131:0/3400233301 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f40740056c0 con 0x7f4084074560 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.359+0000 7f408967f640 1 -- 172.21.15.131:0/3400233301 >> v1:172.21.15.131:6789/0 conn(0x7f4084074560 legacy=0x7f4084102f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:55.119 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 -- 172.21.15.131:0/3400233301 shutdown_connections 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 -- 172.21.15.131:0/3400233301 >> 172.21.15.131:0/3400233301 conn(0x7f40840faaa0 msgr2=0x7f40840fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 -- 172.21.15.131:0/3400233301 shutdown_connections 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 -- 172.21.15.131:0/3400233301 wait complete. 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 Processor -- start 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 -- start start 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f408967f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f408410f340 con 0x7f4084074560 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f4082ffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4084074560 0x7f408410ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41242/0 (socket says 172.21.15.131:41242) 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.360+0000 7f4082ffd640 1 -- 172.21.15.131:0/2435305350 learned_addr learned my addr 172.21.15.131:0/2435305350 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398354770 0 0) 0x7f408410f340 con 0x7f4084074560 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40700053f0 con 0x7f4084074560 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 437024133 0 0) 0x7f40700053f0 con 0x7f4084074560 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f408410f340 con 0x7f4084074560 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4074001430 con 0x7f4084074560 2024-09-10T18:33:55.120 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1973302 0 0) 0x7f408410f340 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f408410f4d0 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408410f7a0 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4084113280 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f4074001b30 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.361+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4074006190 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.362+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7f4074006440 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.362+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4064005040 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.362+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f4074047b60 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.367+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f4074007230 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.523+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 --> v1:172.21.15.131:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f40640052f0 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.526+0000 7f407bfff640 1 -- 172.21.15.131:0/2435305350 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7f4074027fe0 con 0x7f4084074560 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.529+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 >> v1:172.21.15.131:6800/2393239631 conn(0x7f407003a520 legacy=0x7f407003c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.529+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 >> v1:172.21.15.131:6789/0 conn(0x7f4084074560 legacy=0x7f408410ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:55.121 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.529+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 shutdown_connections 2024-09-10T18:33:55.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.529+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 >> 172.21.15.131:0/2435305350 conn(0x7f40840faaa0 msgr2=0x7f4084105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:55.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.529+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 shutdown_connections 2024-09-10T18:33:55.122 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:54.529+0000 7f408967f640 1 -- 172.21.15.131:0/2435305350 wait complete. 2024-09-10T18:33:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2435305350' entity='client.admin' 2024-09-10T18:33:56.636 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd9810640 1 Processor -- start 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd9810640 1 -- start start 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd9810640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1cd40ffdd0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd2ffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1cd40ff2b0 0x7f1cd40ff6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41248/0 (socket says 172.21.15.131:41248) 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd2ffd640 1 -- 172.21.15.131:0/604334198 learned_addr learned my addr 172.21.15.131:0/604334198 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415108214 0 0) 0x7f1cd40ffdd0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc00034e0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2334278210 0 0) 0x7f1cc00034e0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd4104720 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.726+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f1cc4001590 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f1cc4001c00 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd27fc640 1 -- 172.21.15.131:0/604334198 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f1cc40055e0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.637 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd9810640 1 -- 172.21.15.131:0/604334198 >> v1:172.21.15.131:6789/0 conn(0x7f1cd40ff2b0 legacy=0x7f1cd40ff6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd9810640 1 -- 172.21.15.131:0/604334198 shutdown_connections 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd9810640 1 -- 172.21.15.131:0/604334198 >> 172.21.15.131:0/604334198 conn(0x7f1cd40fab10 msgr2=0x7f1cd40fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd9810640 1 -- 172.21.15.131:0/604334198 shutdown_connections 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.727+0000 7f1cd9810640 1 -- 172.21.15.131:0/604334198 wait complete. 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1cd9810640 1 Processor -- start 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1cd9810640 1 -- start start 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1cd9810640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1cd410e240 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1cd2ffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1cd40ff2b0 0x7f1cd410db30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41250/0 (socket says 172.21.15.131:41250) 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1cd2ffd640 1 -- 172.21.15.131:0/1224919994 learned_addr learned my addr 172.21.15.131:0/1224919994 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254644406 0 0) 0x7f1cd410e240 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc0005440 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1651537968 0 0) 0x7f1cc0005440 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1cd410e240 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.638 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f1cc40048c0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2827791266 0 0) 0x7f1cd410e240 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd410e3d0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.728+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd410e6a0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.729+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd4112180 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.729+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f1cc4001800 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.729+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f1cc4006000 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.729+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7f1cc4011300 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.729+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cb4005040 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.730+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f1cc4047ac0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.734+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f1cc4049e60 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.892+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 --> v1:172.21.15.131:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f1cb4005870 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.896+0000 7f1ccb7fe640 1 -- 172.21.15.131:0/1224919994 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7f1cc4012cb0 con 0x7f1cd40ff2b0 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.898+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 >> v1:172.21.15.131:6800/2393239631 conn(0x7f1cc003a770 legacy=0x7f1cc003cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:56.639 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.898+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 >> v1:172.21.15.131:6789/0 conn(0x7f1cd40ff2b0 legacy=0x7f1cd410db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:56.640 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.898+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 shutdown_connections 2024-09-10T18:33:56.640 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.898+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 >> 172.21.15.131:0/1224919994 conn(0x7f1cd40fab10 msgr2=0x7f1cd4104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:56.640 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.898+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 shutdown_connections 2024-09-10T18:33:56.640 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:55.898+0000 7f1cd9810640 1 -- 172.21.15.131:0/1224919994 wait complete. 2024-09-10T18:33:56.640 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling the dashboard module... 2024-09-10T18:33:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1224919994' entity='client.admin' 2024-09-10T18:33:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:56 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:58.046 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2966519789' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:33:58.046 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:57 smithi131 ceph-mon[29369]: from='mgr.14118 v1:172.21.15.131:0/4095653052' entity='mgr.y' 2024-09-10T18:33:58.047 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ignoring --setuser ceph since I am not root 2024-09-10T18:33:58.047 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ignoring --setgroup ceph since I am not root 2024-09-10T18:33:58.047 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.044+0000 7fcf0b940ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:33:58.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.099+0000 7fcf0b940ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:33:58.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.163+0000 7fcf0b940ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:33:58.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.293+0000 7fcf0b940ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:33:58.701 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.434+0000 7fcf0b940ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:33:58.702 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.699+0000 7fcf0b940ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:33:59.001 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2966519789' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:33:59.001 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-mon[29369]: mgrmap e9: y(active, since 18s) 2024-09-10T18:33:59.001 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.764+0000 7fcf0b940ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:33:59.001 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:58.933+0000 7fcf0b940ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:33:59.476 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc0112bd640 1 Processor -- start 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc0112bd640 1 -- start start 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc0112bd640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc00c0ffd70 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc00affd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc00c0ff250 0x7fc00c0ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41256/0 (socket says 172.21.15.131:41256) 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc00affd640 1 -- 172.21.15.131:0/4138853510 learned_addr learned my addr 172.21.15.131:0/4138853510 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248803472 0 0) 0x7fc00c0ffd70 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.122+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff80034e0 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.123+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268680658 0 0) 0x7fbff80034e0 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.123+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c104710 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.123+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbffc001590 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.123+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fbffc001cd0 con 0x7fc00c0ff250 2024-09-10T18:33:59.477 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.123+0000 7fc00a7fc640 1 -- 172.21.15.131:0/4138853510 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbffc0056c0 con 0x7fc00c0ff250 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.124+0000 7fc0112bd640 1 -- 172.21.15.131:0/4138853510 >> v1:172.21.15.131:6789/0 conn(0x7fc00c0ff250 legacy=0x7fc00c0ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.124+0000 7fc0112bd640 1 -- 172.21.15.131:0/4138853510 shutdown_connections 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.124+0000 7fc0112bd640 1 -- 172.21.15.131:0/4138853510 >> 172.21.15.131:0/4138853510 conn(0x7fc00c0fa9d0 msgr2=0x7fc00c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.124+0000 7fc0112bd640 1 -- 172.21.15.131:0/4138853510 shutdown_connections 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.124+0000 7fc0112bd640 1 -- 172.21.15.131:0/4138853510 wait complete. 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0112bd640 1 Processor -- start 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0112bd640 1 -- start start 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0112bd640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc00c10f380 con 0x7fc00c0ff250 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc00affd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc00c0ff250 0x7fc00c10ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:41268/0 (socket says 172.21.15.131:41268) 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc00affd640 1 -- 172.21.15.131:0/2966519789 learned_addr learned my addr 172.21.15.131:0/2966519789 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309475709 0 0) 0x7fc00c10f380 con 0x7fc00c0ff250 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff80053f0 con 0x7fc00c0ff250 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2959937240 0 0) 0x7fbff80053f0 con 0x7fc00c0ff250 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc00c10f380 con 0x7fc00c0ff250 2024-09-10T18:33:59.478 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbffc001430 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.125+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2502375965 0 0) 0x7fc00c10f380 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.126+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c10f510 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.126+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc00c10f7e0 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.126+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc00c113270 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.126+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fbffc001b30 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.126+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbffc0061c0 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.126+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 8) v1 ==== 45457+0+0 (unknown 3967220414 0 0) 0x7fbffc011540 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.127+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfec005040 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.127+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7fbffc048940 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.131+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fbffc016d20 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.337+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fbfec0052f0 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.901+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mgrmap(e 9) v1 ==== 45470+0+0 (unknown 3244635550 0 0) 0x7fbffc013250 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.902+0000 7fc0037fe640 1 -- 172.21.15.131:0/2966519789 <== mon.0 v1:172.21.15.131:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7fbffc048270 con 0x7fc00c0ff250 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.905+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 >> v1:172.21.15.131:6800/2393239631 conn(0x7fbff803a5d0 legacy=0x7fbff803ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:59.479 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.905+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 >> v1:172.21.15.131:6789/0 conn(0x7fc00c0ff250 legacy=0x7fc00c10ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:33:59.480 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.906+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 shutdown_connections 2024-09-10T18:33:59.480 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.906+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 >> 172.21.15.131:0/2966519789 conn(0x7fc00c0fa9d0 msgr2=0x7fc00c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:33:59.480 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.906+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 shutdown_connections 2024-09-10T18:33:59.480 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:57.906+0000 7fc0112bd640 1 -- 172.21.15.131:0/2966519789 wait complete. 2024-09-10T18:33:59.983 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:59 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:59.808+0000 7fcf0b940ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:33:59.983 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:33:59 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:33:59.873+0000 7fcf0b940ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:34:00.501 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1252812995' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:34:00.769 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:00.597+0000 7fcf0b940ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:34:00.769 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:00.708+0000 7fcf0b940ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:34:00.776 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.981+0000 7f6523379640 1 Processor -- start 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6523379640 1 -- start start 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6523379640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f651c104720 con 0x7f651c074480 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6521115640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f651c074480 0x7f651c103000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47676/0 (socket says 172.21.15.131:47676) 2024-09-10T18:34:00.777 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6521115640 1 -- 172.21.15.131:0/2407276276 learned_addr learned my addr 172.21.15.131:0/2407276276 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045722816 0 0) 0x7f651c104720 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65080034e0 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.982+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2320570509 0 0) 0x7f65080034e0 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c1058c0 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f650c001590 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f650c001cd0 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6513fff640 1 -- 172.21.15.131:0/2407276276 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f650c0056c0 con 0x7f651c074480 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6523379640 1 -- 172.21.15.131:0/2407276276 >> v1:172.21.15.131:6789/0 conn(0x7f651c074480 legacy=0x7f651c103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6523379640 1 -- 172.21.15.131:0/2407276276 shutdown_connections 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6523379640 1 -- 172.21.15.131:0/2407276276 >> 172.21.15.131:0/2407276276 conn(0x7f651c0fab10 msgr2=0x7f651c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:00.778 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.983+0000 7f6523379640 1 -- 172.21.15.131:0/2407276276 shutdown_connections 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.984+0000 7f6523379640 1 -- 172.21.15.131:0/2407276276 wait complete. 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.984+0000 7f6523379640 1 Processor -- start 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.984+0000 7f6523379640 1 -- start start 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.984+0000 7f6523379640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f651c10f330 con 0x7f651c074480 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.984+0000 7f6521115640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f651c074480 0x7f651c10ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47686/0 (socket says 172.21.15.131:47686) 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.984+0000 7f6521115640 1 -- 172.21.15.131:0/1252812995 learned_addr learned my addr 172.21.15.131:0/1252812995 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436719100 0 0) 0x7f651c10f330 con 0x7f651c074480 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65080053f0 con 0x7f651c074480 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 945128500 0 0) 0x7f65080053f0 con 0x7f651c074480 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f651c10f330 con 0x7f651c074480 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f650c001430 con 0x7f651c074480 2024-09-10T18:34:00.779 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3992460547 0 0) 0x7f651c10f330 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c10f4c0 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651c10f790 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f650c001b30 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f650c006220 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.985+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f651c113220 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.986+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 9) v1 ==== 45470+0+0 (unknown 3244635550 0 0) 0x7f650c011680 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.986+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7f650c047be0 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.986+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651c108a20 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.986+0000 7f6520914640 1 -- 172.21.15.131:0/1252812995 >> v1:172.21.15.131:6800/2393239631 conn(0x7f650803a660 legacy=0x7f650803cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/2393239631 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:33:59.991+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f650c016e40 con 0x7f651c074480 2024-09-10T18:34:00.780 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.187+0000 7f6520914640 1 -- 172.21.15.131:0/1252812995 >> v1:172.21.15.131:6800/2393239631 conn(0x7f650803a660 legacy=0x7f650803cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/2393239631 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.187+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f651c02aaa0 con 0x7f651c074480 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.187+0000 7f6511ffb640 1 -- 172.21.15.131:0/1252812995 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f650c016840 con 0x7f651c074480 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.190+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 >> v1:172.21.15.131:6800/2393239631 conn(0x7f650803a660 legacy=0x7f650803cb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.190+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 >> v1:172.21.15.131:6789/0 conn(0x7f651c074480 legacy=0x7f651c10ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.190+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 shutdown_connections 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.190+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 >> 172.21.15.131:0/1252812995 conn(0x7f651c0fab10 msgr2=0x7f651c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.190+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 shutdown_connections 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:00.190+0000 7f6523379640 1 -- 172.21.15.131:0/1252812995 wait complete. 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for the mgr to restart... 2024-09-10T18:34:00.781 INFO:teuthology.orchestra.run.smithi131.stdout:Waiting for mgr epoch 9... 2024-09-10T18:34:01.024 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:00.837+0000 7fcf0b940ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:34:01.024 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:00.896+0000 7fcf0b940ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:34:01.024 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.022+0000 7fcf0b940ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:34:01.276 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.095+0000 7fcf0b940ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:34:01.276 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.153+0000 7fcf0b940ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:34:01.556 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.292+0000 7fcf0b940ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:34:01.557 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.355+0000 7fcf0b940ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:34:01.557 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.419+0000 7fcf0b940ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:34:01.557 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.498+0000 7fcf0b940ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:34:01.557 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.554+0000 7fcf0b940ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:34:01.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:01.610+0000 7fcf0b940ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:34:02.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:02.345+0000 7fcf0b940ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:34:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: Active manager daemon y restarted 2024-09-10T18:34:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: Activating manager daemon y 2024-09-10T18:34:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: mgrmap e10: y(active, starting, since 0.00295216s) 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: Manager daemon y is now available 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:34:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:03.372+0000 7fcebc49d640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:34:03] ENGINE Error in HTTPServer.serve 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: Traceback (most recent call last): 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._connections.run(self.expiration_interval) 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._run(expiration_interval) 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:34:03.578 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: s = self.context.wrap_socket( 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: return self.sslsocket_class._create( 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self.do_handshake() 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._sslobj.do_handshake() 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:34:03.579 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:34:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:34:04.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:04.320 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.271+0000 7fbf2295d640 1 Processor -- start 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.271+0000 7fbf2295d640 1 -- start start 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.271+0000 7fbf2295d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbf1c0ffd70 con 0x7fbf1c0ff250 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1bfff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbf1c0ff250 0x7fbf1c0ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47698/0 (socket says 172.21.15.131:47698) 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1bfff640 1 -- 172.21.15.131:0/297721883 learned_addr learned my addr 172.21.15.131:0/297721883 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227010713 0 0) 0x7fbf1c0ffd70 con 0x7fbf1c0ff250 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf080034e0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842014786 0 0) 0x7fbf080034e0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.321 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c104710 con 0x7fbf1c0ff250 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbf0c001590 con 0x7fbf1c0ff250 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fbf0c001cd0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.272+0000 7fbf1affd640 1 -- 172.21.15.131:0/297721883 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbf0c0056c0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.273+0000 7fbf2295d640 1 -- 172.21.15.131:0/297721883 >> v1:172.21.15.131:6789/0 conn(0x7fbf1c0ff250 legacy=0x7fbf1c0ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.273+0000 7fbf2295d640 1 -- 172.21.15.131:0/297721883 shutdown_connections 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.273+0000 7fbf2295d640 1 -- 172.21.15.131:0/297721883 >> 172.21.15.131:0/297721883 conn(0x7fbf1c0fa9d0 msgr2=0x7fbf1c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.273+0000 7fbf2295d640 1 -- 172.21.15.131:0/297721883 shutdown_connections 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.273+0000 7fbf2295d640 1 -- 172.21.15.131:0/297721883 wait complete. 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf2295d640 1 Processor -- start 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf2295d640 1 -- start start 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf2295d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbf1c10f340 con 0x7fbf1c0ff250 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf1bfff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbf1c0ff250 0x7fbf1c10ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47714/0 (socket says 172.21.15.131:47714) 2024-09-10T18:34:04.322 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf1bfff640 1 -- 172.21.15.131:0/1989548345 learned_addr learned my addr 172.21.15.131:0/1989548345 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7335910 0 0) 0x7fbf1c10f340 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.274+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf080053f0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 394908243 0 0) 0x7fbf080053f0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbf1c10f340 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbf0c001430 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4273883730 0 0) 0x7fbf1c10f340 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c10f4d0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c10f7a0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c113280 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fbf0c001b30 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.275+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fbf0c0061c0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.323 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.276+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 9) v1 ==== 45470+0+0 (unknown 3244635550 0 0) 0x7fbf0c006320 con 0x7fbf1c0ff250 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.276+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6800/2393239631 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbefc000d40 con 0x7fbf0803a670 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.276+0000 7fbf1b7fe640 1 -- 172.21.15.131:0/1989548345 >> v1:172.21.15.131:6800/2393239631 conn(0x7fbf0803a670 legacy=0x7fbf0803cb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/2393239631 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.276+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (unknown 3306797590 0 0) 0x7fbf0c047ae0 con 0x7fbf1c0ff250 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.477+0000 7fbf1b7fe640 1 -- 172.21.15.131:0/1989548345 >> v1:172.21.15.131:6800/2393239631 conn(0x7fbf0803a670 legacy=0x7fbf0803cb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/2393239631 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:01.878+0000 7fbf1b7fe640 1 -- 172.21.15.131:0/1989548345 >> v1:172.21.15.131:6800/2393239631 conn(0x7fbf0803a670 legacy=0x7fbf0803cb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/2393239631 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:02.348+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mgrmap(e 10) v1 ==== 45272+0+0 (unknown 3952927234 0 0) 0x7fbf0c046b20 con 0x7fbf1c0ff250 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:02.348+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 >> v1:172.21.15.131:6800/2393239631 conn(0x7fbf0803a670 legacy=0x7fbf0803cb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.349+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mgrmap(e 11) v1 ==== 45364+0+0 (unknown 429455039 0 0) 0x7fbf0c048f60 con 0x7fbf1c0ff250 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.350+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6800/3123609852 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbefc000d40 con 0x7fbf0803eff0 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.351+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (unknown 0 0 4193150809) 0x7fbefc000d40 con 0x7fbf0803eff0 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 --> v1:172.21.15.131:6800/3123609852 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbefc002730 con 0x7fbf0803eff0 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf18ff9640 1 -- 172.21.15.131:0/1989548345 <== mgr.14150 v1:172.21.15.131:6800/3123609852 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7fbefc002730 con 0x7fbf0803eff0 2024-09-10T18:34:04.324 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 >> v1:172.21.15.131:6800/3123609852 conn(0x7fbf0803eff0 legacy=0x7fbf080413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 >> v1:172.21.15.131:6789/0 conn(0x7fbf1c0ff250 legacy=0x7fbf1c10ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 shutdown_connections 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 >> 172.21.15.131:0/1989548345 conn(0x7fbf1c0fa9d0 msgr2=0x7fbf1c072fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 shutdown_connections 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:03.356+0000 7fbf2295d640 1 -- 172.21.15.131:0/1989548345 wait complete. 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:mgr epoch 9 is available 2024-09-10T18:34:04.325 INFO:teuthology.orchestra.run.smithi131.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:34:04.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: [10/Sep/2024:18:34:03] ENGINE Bus STARTING 2024-09-10T18:34:04.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: from='client.14154 v1:172.21.15.131:0/1989548345' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: mgrmap e11: y(active, since 1.00556s) 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: from='client.14154 v1:172.21.15.131:0/1989548345' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: [10/Sep/2024:18:34:03] ENGINE Serving on https://172.21.15.131:7150 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: [10/Sep/2024:18:34:03] ENGINE Bus STARTED 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: [10/Sep/2024:18:34:03] ENGINE Error in HTTPServer.serve 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-10T18:34:04.565 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:34:04.566 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:04.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:04 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:05.560 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:34:05.560 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.744+0000 7fc6deb80640 1 Processor -- start 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.744+0000 7fc6deb80640 1 -- start start 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.744+0000 7fc6deb80640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6d8104720 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.744+0000 7fc6dc91c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc6d8074530 0x7fc6d8103000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47792/0 (socket says 172.21.15.131:47792) 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.744+0000 7fc6dc91c640 1 -- 172.21.15.131:0/4201421368 learned_addr learned my addr 172.21.15.131:0/4201421368 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317841708 0 0) 0x7fc6d8104720 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c80034e0 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878278791 0 0) 0x7fc6c80034e0 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d81058c0 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fc6cc001590 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fc6cc001cd0 con 0x7fc6d8074530 2024-09-10T18:34:05.561 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.745+0000 7fc6d77fe640 1 -- 172.21.15.131:0/4201421368 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fc6cc0056c0 con 0x7fc6d8074530 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.746+0000 7fc6deb80640 1 -- 172.21.15.131:0/4201421368 >> v1:172.21.15.131:6789/0 conn(0x7fc6d8074530 legacy=0x7fc6d8103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.746+0000 7fc6deb80640 1 -- 172.21.15.131:0/4201421368 shutdown_connections 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.746+0000 7fc6deb80640 1 -- 172.21.15.131:0/4201421368 >> 172.21.15.131:0/4201421368 conn(0x7fc6d80fab10 msgr2=0x7fc6d80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.746+0000 7fc6deb80640 1 -- 172.21.15.131:0/4201421368 shutdown_connections 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.746+0000 7fc6deb80640 1 -- 172.21.15.131:0/4201421368 wait complete. 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.746+0000 7fc6deb80640 1 Processor -- start 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6deb80640 1 -- start start 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6deb80640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6d810f300 con 0x7fc6d8074530 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6dc91c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fc6d8074530 0x7fc6d810ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47802/0 (socket says 172.21.15.131:47802) 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6dc91c640 1 -- 172.21.15.131:0/368677617 learned_addr learned my addr 172.21.15.131:0/368677617 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106530317 0 0) 0x7fc6d810f300 con 0x7fc6d8074530 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6c80053f0 con 0x7fc6d8074530 2024-09-10T18:34:05.562 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 467034400 0 0) 0x7fc6c80053f0 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6d810f300 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.747+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fc6cc001430 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.748+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2622199625 0 0) 0x7fc6d810f300 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.748+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6d810f490 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.748+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d810f760 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.748+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d8113240 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.748+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fc6cc001b30 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.748+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fc6cc006100 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.749+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 11) v1 ==== 45364+0+0 (unknown 429455039 0 0) 0x7fc6cc011440 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.749+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc005040 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.749+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7fc6cc048760 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.753+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fc6cc016cc0 con 0x7fc6d8074530 2024-09-10T18:34:05.563 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.908+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mgrmap(e 12) v1 ==== 45470+0+0 (unknown 184107551 0 0) 0x7fc6cc047920 con 0x7fc6d8074530 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.923+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6bc000bc0 con 0x7fc6c803a500 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.989+0000 7fc6d57fa640 1 -- 172.21.15.131:0/368677617 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7fc6bc000bc0 con 0x7fc6c803a500 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.991+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 >> v1:172.21.15.131:6800/3123609852 conn(0x7fc6c803a500 legacy=0x7fc6c803c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.991+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 >> v1:172.21.15.131:6789/0 conn(0x7fc6d8074530 legacy=0x7fc6d810ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.992+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 shutdown_connections 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.992+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 >> 172.21.15.131:0/368677617 conn(0x7fc6d80fab10 msgr2=0x7fc6d8105cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.992+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 shutdown_connections 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:04.992+0000 7fc6deb80640 1 -- 172.21.15.131:0/368677617 wait complete. 2024-09-10T18:34:05.564 INFO:teuthology.orchestra.run.smithi131.stdout:Creating initial admin user... 2024-09-10T18:34:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:05 smithi131 ceph-mon[29369]: mgrmap e12: y(active, since 2s) 2024-09-10T18:34:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:05 smithi131 ceph-mon[29369]: from='client.14162 v1:172.21.15.131:0/368677617' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:07.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$biyEBEWMCnu1RavKZe2XYOz1b31eVl1bfxAvCJyrT52V2B6EecpBK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725993246, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:34:07.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.228+0000 7f57eecfa640 1 Processor -- start 2024-09-10T18:34:07.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57eecfa640 1 -- start start 2024-09-10T18:34:07.246 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57eecfa640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57e8074a50 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57eca96640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f57e8075e10 0x7f57e8074340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47814/0 (socket says 172.21.15.131:47814) 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57eca96640 1 -- 172.21.15.131:0/3894326488 learned_addr learned my addr 172.21.15.131:0/3894326488 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812527660 0 0) 0x7f57e8074a50 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d40034e0 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 97074431 0 0) 0x7f57d40034e0 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8074be0 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.229+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f57d8001910 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f57d8001f80 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57e77fe640 1 -- 172.21.15.131:0/3894326488 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f57d8005960 con 0x7f57e8075e10 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57eecfa640 1 -- 172.21.15.131:0/3894326488 >> v1:172.21.15.131:6789/0 conn(0x7f57e8075e10 legacy=0x7f57e8074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57eecfa640 1 -- 172.21.15.131:0/3894326488 shutdown_connections 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57eecfa640 1 -- 172.21.15.131:0/3894326488 >> 172.21.15.131:0/3894326488 conn(0x7f57e80fbb60 msgr2=0x7f57e80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57eecfa640 1 -- 172.21.15.131:0/3894326488 shutdown_connections 2024-09-10T18:34:07.247 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.230+0000 7f57eecfa640 1 -- 172.21.15.131:0/3894326488 wait complete. 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57eecfa640 1 Processor -- start 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57eecfa640 1 -- start start 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57eecfa640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57e81115e0 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57eca96640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f57e8075e10 0x7f57e8110ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47826/0 (socket says 172.21.15.131:47826) 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57eca96640 1 -- 172.21.15.131:0/1854190012 learned_addr learned my addr 172.21.15.131:0/1854190012 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227959176 0 0) 0x7f57e81115e0 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.231+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57d4005440 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1847492477 0 0) 0x7f57d4005440 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57e81115e0 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f57d8004c40 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1370167651 0 0) 0x7f57e81115e0 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8111770 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e8111a40 con 0x7f57e8075e10 2024-09-10T18:34:07.248 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e8115520 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f57d8001b80 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.232+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f57d80063f0 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.233+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 12) v1 ==== 45470+0+0 (unknown 184107551 0 0) 0x7f57d8011700 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.233+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c8005010 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.233+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7f57d8047fb0 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.237+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f57d8016f30 con 0x7f57e8075e10 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.410+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f57c80023f0 con 0x7f57d403a7b0 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.632+0000 7f57e57fa640 1 -- 172.21.15.131:0/1854190012 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 3587651884) 0x7f57c80023f0 con 0x7f57d403a7b0 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.634+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 >> v1:172.21.15.131:6800/3123609852 conn(0x7f57d403a7b0 legacy=0x7f57d403cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.634+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 >> v1:172.21.15.131:6789/0 conn(0x7f57e8075e10 legacy=0x7f57e8110ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.634+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 shutdown_connections 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.634+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 >> 172.21.15.131:0/1854190012 conn(0x7f57e80fbb60 msgr2=0x7f57e8105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.634+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 shutdown_connections 2024-09-10T18:34:07.249 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:06.634+0000 7f57eecfa640 1 -- 172.21.15.131:0/1854190012 wait complete. 2024-09-10T18:34:07.250 INFO:teuthology.orchestra.run.smithi131.stdout:Fetching dashboard port number... 2024-09-10T18:34:07.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:07 smithi131 ceph-mon[29369]: from='client.14164 v1:172.21.15.131:0/1854190012' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:07.887 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1546701339' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.104+0000 7f7ed54ad640 1 Processor -- start 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.104+0000 7f7ed54ad640 1 -- start start 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.104+0000 7f7ed54ad640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ed0104660 con 0x7f7ed0073c70 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7eceffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7ed0073c70 0x7f7ed0102f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47834/0 (socket says 172.21.15.131:47834) 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7eceffd640 1 -- 172.21.15.131:0/808097085 learned_addr learned my addr 172.21.15.131:0/808097085 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343735226 0 0) 0x7f7ed0104660 con 0x7f7ed0073c70 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb80034e0 con 0x7f7ed0073c70 2024-09-10T18:34:08.881 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050785801 0 0) 0x7f7eb80034e0 con 0x7f7ed0073c70 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed0105800 con 0x7f7ed0073c70 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7ec0001910 con 0x7f7ed0073c70 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f7ec0002050 con 0x7f7ed0073c70 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.105+0000 7f7ecdffb640 1 -- 172.21.15.131:0/808097085 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7ec00059d0 con 0x7f7ed0073c70 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 -- 172.21.15.131:0/808097085 >> v1:172.21.15.131:6789/0 conn(0x7f7ed0073c70 legacy=0x7f7ed0102f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 -- 172.21.15.131:0/808097085 shutdown_connections 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 -- 172.21.15.131:0/808097085 >> 172.21.15.131:0/808097085 conn(0x7f7ed00fa9d0 msgr2=0x7f7ed00fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 -- 172.21.15.131:0/808097085 shutdown_connections 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 -- 172.21.15.131:0/808097085 wait complete. 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 Processor -- start 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.106+0000 7f7ed54ad640 1 -- start start 2024-09-10T18:34:08.882 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7ed54ad640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7ed010f250 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eceffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7ed0073c70 0x7f7ed010eb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:47840/0 (socket says 172.21.15.131:47840) 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eceffd640 1 -- 172.21.15.131:0/1546701339 learned_addr learned my addr 172.21.15.131:0/1546701339 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1028128620 0 0) 0x7f7ed010f250 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7eb80053f0 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 737770407 0 0) 0x7f7eb80053f0 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ed010f250 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7ec0005790 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1548156266 0 0) 0x7f7ed010f250 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed010f3e0 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed010f6b0 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed0113190 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f7ec0004ee0 con 0x7f7ed0073c70 2024-09-10T18:34:08.883 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.107+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f7ec0006460 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.108+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 12) v1 ==== 45470+0+0 (unknown 184107551 0 0) 0x7f7ec0011830 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.108+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ebc005040 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.108+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7f7ec0048010 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.113+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7ec00170c0 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.267+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f7ebc005870 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.268+0000 7f7eb77fe640 1 -- 172.21.15.131:0/1546701339 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f7ec0016ac0 con 0x7f7ed0073c70 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.270+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 >> v1:172.21.15.131:6800/3123609852 conn(0x7f7eb803a610 legacy=0x7f7eb803cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.270+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 >> v1:172.21.15.131:6789/0 conn(0x7f7ed0073c70 legacy=0x7f7ed010eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.270+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 shutdown_connections 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.270+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 >> 172.21.15.131:0/1546701339 conn(0x7f7ed00fa9d0 msgr2=0x7f7ed0105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.270+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 shutdown_connections 2024-09-10T18:34:08.884 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:08.270+0000 7f7ed54ad640 1 -- 172.21.15.131:0/1546701339 wait complete. 2024-09-10T18:34:08.888 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:34:08.889 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout disabled 2024-09-10T18:34:08.896 INFO:teuthology.orchestra.run.smithi131.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:34:08.896 INFO:teuthology.orchestra.run.smithi131.stdout:systemctl: stdout inactive 2024-09-10T18:34:08.896 INFO:teuthology.orchestra.run.smithi131.stdout:firewalld.service is not enabled 2024-09-10T18:34:08.896 INFO:teuthology.orchestra.run.smithi131.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:34:08.898 INFO:teuthology.orchestra.run.smithi131.stdout:Ceph Dashboard is now available at: 2024-09-10T18:34:08.898 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:08.898 INFO:teuthology.orchestra.run.smithi131.stdout: URL: https://smithi131.front.sepia.ceph.com:8443/ 2024-09-10T18:34:08.898 INFO:teuthology.orchestra.run.smithi131.stdout: User: admin 2024-09-10T18:34:08.898 INFO:teuthology.orchestra.run.smithi131.stdout: Password: 4skn92hntd 2024-09-10T18:34:08.898 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:08.899 INFO:teuthology.orchestra.run.smithi131.stdout:Saving cluster configuration to /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config directory 2024-09-10T18:34:08.899 INFO:teuthology.orchestra.run.smithi131.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:34:09.813 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:09 smithi131 ceph-mon[29369]: mgrmap e13: y(active, since 6s) 2024-09-10T18:34:10.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c5034b640 1 Processor -- start 2024-09-10T18:34:10.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c5034b640 1 -- start start 2024-09-10T18:34:10.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c5034b640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c480769e0 con 0x7f0c480765d0 2024-09-10T18:34:10.097 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c4e0e7640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0c480765d0 0x7f0c48074b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54864/0 (socket says 172.21.15.131:54864) 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c4e0e7640 1 -- 172.21.15.131:0/2825403754 learned_addr learned my addr 172.21.15.131:0/2825403754 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347706873 0 0) 0x7f0c480769e0 con 0x7f0c480765d0 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.353+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c300034e0 con 0x7f0c480765d0 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 869803856 0 0) 0x7f0c300034e0 con 0x7f0c480765d0 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48075210 con 0x7f0c480765d0 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0c40001910 con 0x7f0c480765d0 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0c40002050 con 0x7f0c480765d0 2024-09-10T18:34:10.098 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c4d0e5640 1 -- 172.21.15.131:0/2825403754 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0c40005a40 con 0x7f0c480765d0 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c5034b640 1 -- 172.21.15.131:0/2825403754 >> v1:172.21.15.131:6789/0 conn(0x7f0c480765d0 legacy=0x7f0c48074b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c5034b640 1 -- 172.21.15.131:0/2825403754 shutdown_connections 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c5034b640 1 -- 172.21.15.131:0/2825403754 >> 172.21.15.131:0/2825403754 conn(0x7f0c480fbb90 msgr2=0x7f0c480fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c5034b640 1 -- 172.21.15.131:0/2825403754 shutdown_connections 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.354+0000 7f0c5034b640 1 -- 172.21.15.131:0/2825403754 wait complete. 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c5034b640 1 Processor -- start 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c5034b640 1 -- start start 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c5034b640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c48111570 con 0x7f0c480765d0 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c4e0e7640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0c480765d0 0x7f0c48110e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54868/0 (socket says 172.21.15.131:54868) 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c4e0e7640 1 -- 172.21.15.131:0/3303144603 learned_addr learned my addr 172.21.15.131:0/3303144603 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233740985 0 0) 0x7f0c48111570 con 0x7f0c480765d0 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.355+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c300053f0 con 0x7f0c480765d0 2024-09-10T18:34:10.099 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 603672932 0 0) 0x7f0c300053f0 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c48111570 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0c400017b0 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4283582383 0 0) 0x7f0c48111570 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c48111700 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c481119d0 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c481154b0 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0c40001580 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.356+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0c40006370 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.357+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f0c40011730 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.357+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c34005040 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.357+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7f0c400480d0 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.361+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0c40013240 con 0x7f0c480765d0 2024-09-10T18:34:10.100 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.517+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 --> v1:172.21.15.131:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0c34005870 con 0x7f0c480765d0 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.518+0000 7f0c46ffd640 1 -- 172.21.15.131:0/3303144603 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7f0c40016ae0 con 0x7f0c480765d0 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.520+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 >> v1:172.21.15.131:6800/3123609852 conn(0x7f0c3003a710 legacy=0x7f0c3003cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.520+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 >> v1:172.21.15.131:6789/0 conn(0x7f0c480765d0 legacy=0x7f0c48110e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.520+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 shutdown_connections 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.520+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 >> 172.21.15.131:0/3303144603 conn(0x7f0c480fbb90 msgr2=0x7f0c48107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.520+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 shutdown_connections 2024-09-10T18:34:10.101 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:09.520+0000 7f0c5034b640 1 -- 172.21.15.131:0/3303144603 wait complete. 2024-09-10T18:34:11.395 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.551+0000 7f4e0ac1d640 1 Processor -- start 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.551+0000 7f4e0ac1d640 1 -- start start 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.551+0000 7f4e0ac1d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e040ffd70 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.551+0000 7f4e089b9640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4e040ff250 0x7f4e040ff660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54882/0 (socket says 172.21.15.131:54882) 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.551+0000 7f4e089b9640 1 -- 172.21.15.131:0/2859850448 learned_addr learned my addr 172.21.15.131:0/2859850448 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168316223 0 0) 0x7f4e040ffd70 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df00034e0 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131297156 0 0) 0x7f4df00034e0 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04104710 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4dfc001590 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 <== mon.0 v1:172.21.15.131:6789/0 4 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f4dfc001cd0 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e037fe640 1 -- 172.21.15.131:0/2859850448 <== mon.0 v1:172.21.15.131:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4dfc0056c0 con 0x7f4e040ff250 2024-09-10T18:34:11.396 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/2859850448 >> v1:172.21.15.131:6789/0 conn(0x7f4e040ff250 legacy=0x7f4e040ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/2859850448 shutdown_connections 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.552+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/2859850448 >> 172.21.15.131:0/2859850448 conn(0x7f4e040fa9d0 msgr2=0x7f4e040fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/2859850448 shutdown_connections 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/2859850448 wait complete. 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e0ac1d640 1 Processor -- start 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e0ac1d640 1 -- start start 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e0ac1d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4e0410e280 con 0x7f4e040ff250 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e089b9640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f4e040ff250 0x7f4e0410db70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54888/0 (socket says 172.21.15.131:54888) 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.553+0000 7f4e089b9640 1 -- 172.21.15.131:0/1983918744 learned_addr learned my addr 172.21.15.131:0/1983918744 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628747414 0 0) 0x7f4e0410e280 con 0x7f4e040ff250 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df00053f0 con 0x7f4e040ff250 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1782133005 0 0) 0x7f4df00053f0 con 0x7f4e040ff250 2024-09-10T18:34:11.397 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4e0410e280 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4dfc001430 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 797052618 0 0) 0x7f4e0410e280 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0410e410 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0410e6e0 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e04112170 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f4dfc001b30 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.554+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f4dfc0060e0 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.555+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f4dfc0114a0 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.555+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de4005040 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.555+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7f4dfc047e20 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.560+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f4dfc012d40 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.797+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 --> v1:172.21.15.131:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f4de4005870 con 0x7f4e040ff250 2024-09-10T18:34:11.398 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.800+0000 7f4e017fa640 1 -- 172.21.15.131:0/1983918744 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (unknown 1169358022 0 0) 0x7f4dfc016790 con 0x7f4e040ff250 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.802+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 >> v1:172.21.15.131:6800/3123609852 conn(0x7f4df003a710 legacy=0x7f4df003cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.802+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 >> v1:172.21.15.131:6789/0 conn(0x7f4e040ff250 legacy=0x7f4e0410db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.802+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 shutdown_connections 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.802+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 >> 172.21.15.131:0/1983918744 conn(0x7f4e040fa9d0 msgr2=0x7f4e04104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.802+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 shutdown_connections 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:/usr/bin/ceph: stderr 2024-09-10T18:34:10.802+0000 7f4e0ac1d640 1 -- 172.21.15.131:0/1983918744 wait complete. 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /sbin/cephadm shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.399 INFO:teuthology.orchestra.run.smithi131.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: sudo /sbin/cephadm shell 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: ceph telemetry on 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout:For more information see: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:11.400 INFO:teuthology.orchestra.run.smithi131.stdout:Bootstrap complete. 2024-09-10T18:34:11.428 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:34:11.428 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:34:11.428 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:34:11.444 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:34:11.444 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:34:11.444 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:34:11.501 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:34:11.501 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:34:11.501 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:34:11.566 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:34:11.566 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:34:11.566 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:34:11.622 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:34:11.623 DEBUG:teuthology.orchestra.run.smithi131:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIKlXr8lmOQ8xdiCFBGPnUqPnHrCElXrtJ2l3sDjoM1lewhJ7uYeU/5qSIrRWTlFZbZFGXIsUH/cTo5iVkgii8ZJ/KMKAP8aRjtb9xEQ9kVqwi7sb+IPHD+bm0+CQXEMSnJ5ooqFpIcfBB17K+0rCSzpjykpAvaqz8+hND4LHqIU7BHzMwrVchmrHdunT7i8TfYWSdguQLwVYXI93SEQJOGO1EsK+Mm9ntASz+lcQE0OcXYrhN6o1eiJ8eL34+NeEyj7f1eiBmzEfLDqNr10rpDKBTP3BEu6kw3qyNUYCEvbm9E2e4DxWdiN1vbi4rshQLQuIqSICYw+SKwQaMjDxEMSxZINQvdN4ua5ciDr+wJ6U+90pFnfg2NuANrgMOazMmPxCQ8KLfp48sjTfOuFvPPCTT+etvi0Wa6rVfJomBYH/mA1Km+y5JgDZ72lTI7APbnwqIyVZi2oUftSZB+0zOUahXgITOlDJhCybrLhnjVEEMJia2oGvPq2PIytE3/8E= ceph-0846fc60-6fa3-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:34:11.709 INFO:teuthology.orchestra.run.smithi131.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIKlXr8lmOQ8xdiCFBGPnUqPnHrCElXrtJ2l3sDjoM1lewhJ7uYeU/5qSIrRWTlFZbZFGXIsUH/cTo5iVkgii8ZJ/KMKAP8aRjtb9xEQ9kVqwi7sb+IPHD+bm0+CQXEMSnJ5ooqFpIcfBB17K+0rCSzpjykpAvaqz8+hND4LHqIU7BHzMwrVchmrHdunT7i8TfYWSdguQLwVYXI93SEQJOGO1EsK+Mm9ntASz+lcQE0OcXYrhN6o1eiJ8eL34+NeEyj7f1eiBmzEfLDqNr10rpDKBTP3BEu6kw3qyNUYCEvbm9E2e4DxWdiN1vbi4rshQLQuIqSICYw+SKwQaMjDxEMSxZINQvdN4ua5ciDr+wJ6U+90pFnfg2NuANrgMOazMmPxCQ8KLfp48sjTfOuFvPPCTT+etvi0Wa6rVfJomBYH/mA1Km+y5JgDZ72lTI7APbnwqIyVZi2oUftSZB+0zOUahXgITOlDJhCybrLhnjVEEMJia2oGvPq2PIytE3/8E= ceph-0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:11.721 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIKlXr8lmOQ8xdiCFBGPnUqPnHrCElXrtJ2l3sDjoM1lewhJ7uYeU/5qSIrRWTlFZbZFGXIsUH/cTo5iVkgii8ZJ/KMKAP8aRjtb9xEQ9kVqwi7sb+IPHD+bm0+CQXEMSnJ5ooqFpIcfBB17K+0rCSzpjykpAvaqz8+hND4LHqIU7BHzMwrVchmrHdunT7i8TfYWSdguQLwVYXI93SEQJOGO1EsK+Mm9ntASz+lcQE0OcXYrhN6o1eiJ8eL34+NeEyj7f1eiBmzEfLDqNr10rpDKBTP3BEu6kw3qyNUYCEvbm9E2e4DxWdiN1vbi4rshQLQuIqSICYw+SKwQaMjDxEMSxZINQvdN4ua5ciDr+wJ6U+90pFnfg2NuANrgMOazMmPxCQ8KLfp48sjTfOuFvPPCTT+etvi0Wa6rVfJomBYH/mA1Km+y5JgDZ72lTI7APbnwqIyVZi2oUftSZB+0zOUahXgITOlDJhCybrLhnjVEEMJia2oGvPq2PIytE3/8E= ceph-0846fc60-6fa3-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:34:11.774 INFO:teuthology.orchestra.run.smithi175.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIKlXr8lmOQ8xdiCFBGPnUqPnHrCElXrtJ2l3sDjoM1lewhJ7uYeU/5qSIrRWTlFZbZFGXIsUH/cTo5iVkgii8ZJ/KMKAP8aRjtb9xEQ9kVqwi7sb+IPHD+bm0+CQXEMSnJ5ooqFpIcfBB17K+0rCSzpjykpAvaqz8+hND4LHqIU7BHzMwrVchmrHdunT7i8TfYWSdguQLwVYXI93SEQJOGO1EsK+Mm9ntASz+lcQE0OcXYrhN6o1eiJ8eL34+NeEyj7f1eiBmzEfLDqNr10rpDKBTP3BEu6kw3qyNUYCEvbm9E2e4DxWdiN1vbi4rshQLQuIqSICYw+SKwQaMjDxEMSxZINQvdN4ua5ciDr+wJ6U+90pFnfg2NuANrgMOazMmPxCQ8KLfp48sjTfOuFvPPCTT+etvi0Wa6rVfJomBYH/mA1Km+y5JgDZ72lTI7APbnwqIyVZi2oUftSZB+0zOUahXgITOlDJhCybrLhnjVEEMJia2oGvPq2PIytE3/8E= ceph-0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:11.787 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:34:12.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1983918744' entity='client.admin' 2024-09-10T18:34:12.163 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:34:14.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:14.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:14.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi131", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:34:14.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:14.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:14.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.955+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/1523204259 >> v1:172.21.15.131:6789/0 conn(0x7f2b280fafd0 legacy=0x7f2b280fb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.955+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/1523204259 shutdown_connections 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.955+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/1523204259 >> 172.21.15.131:0/1523204259 conn(0x7f2b280f67b0 msgr2=0x7f2b280f8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.955+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/1523204259 shutdown_connections 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.955+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/1523204259 wait complete. 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.955+0000 7f2b2e6f9640 1 Processor -- start 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b2e6f9640 1 -- start start 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b2e6f9640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2b280f86d0 con 0x7f2b280fafd0 2024-09-10T18:34:14.969 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b27fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f2b280fafd0 0x7f2b280f6ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54904/0 (socket says 172.21.15.131:54904) 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b27fff640 1 -- 172.21.15.131:0/2817643149 learned_addr learned my addr 172.21.15.131:0/2817643149 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647235472 0 0) 0x7f2b280f86d0 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b10005440 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3018978171 0 0) 0x7f2b10005440 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b280f86d0 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.956+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f2b18002020 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.957+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1262882547 0 0) 0x7f2b280f86d0 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.957+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280f8860 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.957+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b280f8b10 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.957+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b280f75d0 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.957+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f2b180026e0 con 0x7f2b280fafd0 2024-09-10T18:34:14.970 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.957+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f2b18003610 con 0x7f2b280fafd0 2024-09-10T18:34:14.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.958+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f2b18003770 con 0x7f2b280fafd0 2024-09-10T18:34:14.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.958+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b14005040 con 0x7f2b280fafd0 2024-09-10T18:34:14.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.958+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7f2b18047a00 con 0x7f2b280fafd0 2024-09-10T18:34:14.971 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:14.963+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2b18003b60 con 0x7f2b280fafd0 2024-09-10T18:34:15.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.134+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 --> v1:172.21.15.131:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2b14005870 con 0x7f2b280fafd0 2024-09-10T18:34:15.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.137+0000 7f2b24ff9640 1 -- 172.21.15.131:0/2817643149 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7f2b180472e0 con 0x7f2b280fafd0 2024-09-10T18:34:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.139+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 >> v1:172.21.15.131:6800/3123609852 conn(0x7f2b1003a840 legacy=0x7f2b1003ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.139+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 >> v1:172.21.15.131:6789/0 conn(0x7f2b280fafd0 legacy=0x7f2b280f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.139+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 shutdown_connections 2024-09-10T18:34:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.139+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 >> 172.21.15.131:0/2817643149 conn(0x7f2b280f67b0 msgr2=0x7f2b280fd750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.139+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 shutdown_connections 2024-09-10T18:34:15.141 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:15.139+0000 7f2b2e6f9640 1 -- 172.21.15.131:0/2817643149 wait complete. 2024-09-10T18:34:15.714 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:34:15.715 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:34:16.287 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:34:16.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2817643149' entity='client.admin' 2024-09-10T18:34:16.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:16 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:16.408 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:16 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:17.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 -- 172.21.15.131:0/917992626 >> v1:172.21.15.131:6789/0 conn(0x7fa2f40fb5d0 legacy=0x7fa2f40fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:17.753 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 -- 172.21.15.131:0/917992626 shutdown_connections 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 -- 172.21.15.131:0/917992626 >> 172.21.15.131:0/917992626 conn(0x7fa2f40f5f40 msgr2=0x7fa2f40f8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 -- 172.21.15.131:0/917992626 shutdown_connections 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 -- 172.21.15.131:0/917992626 wait complete. 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 Processor -- start 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.750+0000 7fa2fae64640 1 -- start start 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2fae64640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa2f410bec0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f8c00640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fa2f40fb5d0 0x7fa2f410b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54922/0 (socket says 172.21.15.131:54922) 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f8c00640 1 -- 172.21.15.131:0/827514395 learned_addr learned my addr 172.21.15.131:0/827514395 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415822280 0 0) 0x7fa2f410bec0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2dc0053f0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.754 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2784846583 0 0) 0x7fa2dc0053f0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa2f410bec0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa2e4001430 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2869708023 0 0) 0x7fa2f410bec0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f410c050 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.751+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f410c320 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.752+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f410ff30 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.752+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fa2e40024a0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.755 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.752+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fa2e40035e0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.753+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7fa2e4003760 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.753+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e0005040 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.756 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.753+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7fa2e40479e0 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.759 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.757+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fa2e4003b50 con 0x7fa2f40fb5d0 2024-09-10T18:34:17.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.915+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2e0000bc0 con 0x7fa2dc03a850 2024-09-10T18:34:17.920 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.918+0000 7fa2f17fa640 1 -- 172.21.15.131:0/827514395 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7fa2e0000bc0 con 0x7fa2dc03a850 2024-09-10T18:34:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.920+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 >> v1:172.21.15.131:6800/3123609852 conn(0x7fa2dc03a850 legacy=0x7fa2dc03cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.920+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 >> v1:172.21.15.131:6789/0 conn(0x7fa2f40fb5d0 legacy=0x7fa2f410b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:17.922 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.920+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 shutdown_connections 2024-09-10T18:34:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.920+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 >> 172.21.15.131:0/827514395 conn(0x7fa2f40f5f40 msgr2=0x7fa2f40fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.920+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 shutdown_connections 2024-09-10T18:34:17.923 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:17.921+0000 7fa2fae64640 1 -- 172.21.15.131:0/827514395 wait complete. 2024-09-10T18:34:18.736 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi175 2024-09-10T18:34:18.736 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:34:18.736 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:34:18.756 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:34:18.756 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:18.814 INFO:tasks.cephadm:Adding host smithi175 to orchestrator... 2024-09-10T18:34:18.814 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch host add smithi175 2024-09-10T18:34:19.050 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='client.14174 v1:172.21.15.131:0/827514395' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:19 smithi131 ceph-mon[29369]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:19 smithi131 ceph-mon[29369]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:20.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:19 smithi131 ceph-mon[29369]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 -- 172.21.15.131:0/2660277614 >> v1:172.21.15.131:6789/0 conn(0x7efd44102100 legacy=0x7efd441024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 -- 172.21.15.131:0/2660277614 shutdown_connections 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 -- 172.21.15.131:0/2660277614 >> 172.21.15.131:0/2660277614 conn(0x7efd440f7cd0 msgr2=0x7efd440fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 -- 172.21.15.131:0/2660277614 shutdown_connections 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 -- 172.21.15.131:0/2660277614 wait complete. 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 Processor -- start 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.630+0000 7efd4c516640 1 -- start start 2024-09-10T18:34:20.634 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd4c516640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efd44111da0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd4a2b2640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7efd44102100 0x7efd44111690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45504/0 (socket says 172.21.15.131:45504) 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd4a2b2640 1 -- 172.21.15.131:0/2836019318 learned_addr learned my addr 172.21.15.131:0/2836019318 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804356012 0 0) 0x7efd44111da0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd300053f0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4248582427 0 0) 0x7efd300053f0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd44111da0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7efd40002020 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3526305641 0 0) 0x7efd44111da0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd44111f30 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd44112200 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd44114690 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7efd400015d0 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.631+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7efd40003610 con 0x7efd44102100 2024-09-10T18:34:20.635 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.632+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7efd40003790 con 0x7efd44102100 2024-09-10T18:34:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.632+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7efd40048710 con 0x7efd44102100 2024-09-10T18:34:20.636 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.632+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd28005010 con 0x7efd44102100 2024-09-10T18:34:20.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.637+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7efd40003b80 con 0x7efd44102100 2024-09-10T18:34:20.797 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:20.795+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}) v1 -- 0x7efd28000b90 con 0x7efd3003a850 2024-09-10T18:34:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:21 smithi131 ceph-mon[29369]: from='client.14176 v1:172.21.15.131:0/2836019318' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi175", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:22 smithi131 ceph-mon[29369]: Deploying cephadm binary to smithi175 2024-09-10T18:34:23.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.434+0000 7efd3affd640 1 -- 172.21.15.131:0/2836019318 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 3916442984) 0x7efd28000b90 con 0x7efd3003a850 2024-09-10T18:34:23.436 INFO:teuthology.orchestra.run.smithi131.stdout:Added host 'smithi175' with addr '172.21.15.175' 2024-09-10T18:34:23.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.436+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 >> v1:172.21.15.131:6800/3123609852 conn(0x7efd3003a850 legacy=0x7efd3003cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:23.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.436+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 >> v1:172.21.15.131:6789/0 conn(0x7efd44102100 legacy=0x7efd44111690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:23.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.436+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 shutdown_connections 2024-09-10T18:34:23.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.436+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 >> 172.21.15.131:0/2836019318 conn(0x7efd440f7cd0 msgr2=0x7efd44102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:23.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.436+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 shutdown_connections 2024-09-10T18:34:23.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:23.436+0000 7efd4c516640 1 -- 172.21.15.131:0/2836019318 wait complete. 2024-09-10T18:34:23.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:23 smithi131 ceph-mon[29369]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:24.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:34:24.298 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:34:24.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:24.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:24 smithi131 ceph-mon[29369]: Added host smithi175 2024-09-10T18:34:24.701 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:24.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:24.702 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:25 smithi131 ceph-mon[29369]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:25 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:25.916 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.875+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/1423473367 >> v1:172.21.15.131:6789/0 conn(0x7fb7f4100ff0 legacy=0x7fb7f41013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.875+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/1423473367 shutdown_connections 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.875+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/1423473367 >> 172.21.15.131:0/1423473367 conn(0x7fb7f4070fb0 msgr2=0x7fb7f40713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.875+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/1423473367 shutdown_connections 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.875+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/1423473367 wait complete. 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.875+0000 7fb7fa5fa640 1 Processor -- start 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7fa5fa640 1 -- start start 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7fa5fa640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7f4111da0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f3fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb7f4100ff0 0x7fb7f4111690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45532/0 (socket says 172.21.15.131:45532) 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f3fff640 1 -- 172.21.15.131:0/3284810892 learned_addr learned my addr 172.21.15.131:0/3284810892 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:25.917 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899208728 0 0) 0x7fb7f4111da0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7dc0053f0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1811708653 0 0) 0x7fb7dc0053f0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7f4111da0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fb7e4002020 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2901340721 0 0) 0x7fb7f4111da0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f4111f30 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f4112200 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f4114720 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fb7e40015d0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.876+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7fb7e40035d0 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.877+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7fb7e4003750 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.877+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e0005010 con 0x7fb7f4100ff0 2024-09-10T18:34:25.918 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.877+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7fb7e4047a50 con 0x7fb7f4100ff0 2024-09-10T18:34:25.919 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:25.882+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb7e4003b90 con 0x7fb7f4100ff0 2024-09-10T18:34:26.045 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.043+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb7e0000b90 con 0x7fb7dc03a850 2024-09-10T18:34:26.046 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.044+0000 7fb7f0ff9640 1 -- 172.21.15.131:0/3284810892 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (unknown 0 0 3030193784) 0x7fb7e0000b90 con 0x7fb7dc03a850 2024-09-10T18:34:26.046 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:34:26.046 INFO:teuthology.orchestra.run.smithi131.stdout:[{"addr": "172.21.15.131", "hostname": "smithi131", "labels": [], "status": ""}, {"addr": "172.21.15.175", "hostname": "smithi175", "labels": [], "status": ""}] 2024-09-10T18:34:26.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.046+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 >> v1:172.21.15.131:6800/3123609852 conn(0x7fb7dc03a850 legacy=0x7fb7dc03cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:26.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.046+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 >> v1:172.21.15.131:6789/0 conn(0x7fb7f4100ff0 legacy=0x7fb7f4111690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:26.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.046+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 shutdown_connections 2024-09-10T18:34:26.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.046+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 >> 172.21.15.131:0/3284810892 conn(0x7fb7f4070fb0 msgr2=0x7fb7f4064930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:26.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.046+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 shutdown_connections 2024-09-10T18:34:26.049 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:26.046+0000 7fb7fa5fa640 1 -- 172.21.15.131:0/3284810892 wait complete. 2024-09-10T18:34:26.677 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:34:26.678 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:34:26.912 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:34:27.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='client.14178 v1:172.21.15.131:0/3284810892' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:34:27.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:27.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:27.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:27.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:27.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:27.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi175", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:34:27.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:27.970 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:28.539 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.504+0000 7f0b14465640 1 -- 172.21.15.131:0/3581952981 >> v1:172.21.15.131:6789/0 conn(0x7f0b0c100ff0 legacy=0x7f0b0c1013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.504+0000 7f0b14465640 1 -- 172.21.15.131:0/3581952981 shutdown_connections 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.504+0000 7f0b14465640 1 -- 172.21.15.131:0/3581952981 >> 172.21.15.131:0/3581952981 conn(0x7f0b0c070fb0 msgr2=0x7f0b0c0713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.504+0000 7f0b14465640 1 -- 172.21.15.131:0/3581952981 shutdown_connections 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.504+0000 7f0b14465640 1 -- 172.21.15.131:0/3581952981 wait complete. 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b14465640 1 Processor -- start 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b14465640 1 -- start start 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b14465640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b0c10cda0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b12201640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0b0c100ff0 0x7f0b0c10c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45542/0 (socket says 172.21.15.131:45542) 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b12201640 1 -- 172.21.15.131:0/2590465671 learned_addr learned my addr 172.21.15.131:0/2590465671 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290786823 0 0) 0x7f0b0c10cda0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.540 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.505+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0af80053f0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1592711681 0 0) 0x7f0af80053f0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b0c10cda0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0b08002020 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2697863197 0 0) 0x7f0b0c10cda0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b0c10cf30 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c10d1e0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c114780 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0b080015d0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.506+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0b080035d0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.507+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f0b08003810 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.507+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0005010 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.507+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (unknown 3737577793 0 0) 0x7f0b08047b70 con 0x7f0b0c100ff0 2024-09-10T18:34:28.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.512+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0b08016ae0 con 0x7f0b0c100ff0 2024-09-10T18:34:28.680 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:28.678+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0af00052c0 con 0x7f0b0c100ff0 2024-09-10T18:34:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:28 smithi131 ceph-mon[29369]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:34:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:28 smithi131 ceph-mon[29369]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:28 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:28 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:28 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2590465671' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:34:29.407 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.405+0000 7f0b02ffd640 1 -- 172.21.15.131:0/2590465671 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7f0b080164e0 con 0x7f0b0c100ff0 2024-09-10T18:34:29.408 INFO:teuthology.orchestra.run.smithi131.stderr:adjusted tunables profile to default 2024-09-10T18:34:29.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.407+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 >> v1:172.21.15.131:6800/3123609852 conn(0x7f0af803a780 legacy=0x7f0af803cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:29.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.407+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 >> v1:172.21.15.131:6789/0 conn(0x7f0b0c100ff0 legacy=0x7f0b0c10c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:29.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.407+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 shutdown_connections 2024-09-10T18:34:29.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.407+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 >> 172.21.15.131:0/2590465671 conn(0x7f0b0c070fb0 msgr2=0x7f0b0c0f8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:29.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.407+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 shutdown_connections 2024-09-10T18:34:29.409 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:29.407+0000 7f0b14465640 1 -- 172.21.15.131:0/2590465671 wait complete. 2024-09-10T18:34:29.976 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:29 smithi131 ceph-mon[29369]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:29.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:29 smithi131 ceph-mon[29369]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:34:29.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:29 smithi131 ceph-mon[29369]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:29.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2590465671' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:34:29.977 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:29 smithi131 ceph-mon[29369]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:30.001 INFO:tasks.cephadm:Adding mon.a on smithi131 2024-09-10T18:34:30.002 INFO:tasks.cephadm:Adding mon.c on smithi131 2024-09-10T18:34:30.002 INFO:tasks.cephadm:Adding mon.b on smithi175 2024-09-10T18:34:30.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b' 2024-09-10T18:34:30.245 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:31.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.694+0000 7f0da4af2640 1 -- 172.21.15.175:0/3940573103 >> v1:172.21.15.131:6789/0 conn(0x7f0da0101000 legacy=0x7f0da01013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:31.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.694+0000 7f0da4af2640 1 -- 172.21.15.175:0/3940573103 shutdown_connections 2024-09-10T18:34:31.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.694+0000 7f0da4af2640 1 -- 172.21.15.175:0/3940573103 >> 172.21.15.175:0/3940573103 conn(0x7f0da00f6bf0 msgr2=0x7f0da00f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:31.699 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.694+0000 7f0da4af2640 1 -- 172.21.15.175:0/3940573103 shutdown_connections 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.694+0000 7f0da4af2640 1 -- 172.21.15.175:0/3940573103 wait complete. 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0da4af2640 1 Processor -- start 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0da4af2640 1 -- start start 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0da4af2640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0da010ce80 con 0x7f0da0101000 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0d9e59c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f0da0101000 0x7f0da010c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.175:37952/0 (socket says 172.21.15.175:37952) 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0d9e59c640 1 -- 172.21.15.175:0/3894973371 learned_addr learned my addr 172.21.15.175:0/3894973371 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4195879432 0 0) 0x7f0da010ce80 con 0x7f0da0101000 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.695+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d8c0053f0 con 0x7f0da0101000 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3750241601 0 0) 0x7f0d8c0053f0 con 0x7f0da0101000 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0da010ce80 con 0x7f0da0101000 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0d90002020 con 0x7f0da0101000 2024-09-10T18:34:31.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2380438005 0 0) 0x7f0da010ce80 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da010d010 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da010d2c0 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da01147e0 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0d900015d0 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.696+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f0d900035d0 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.697+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d80005040 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.697+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f0d90003750 con 0x7f0da0101000 2024-09-10T18:34:31.701 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.697+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7f0d90047a90 con 0x7f0da0101000 2024-09-10T18:34:31.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.702+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0d90003b90 con 0x7f0da0101000 2024-09-10T18:34:31.869 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.868+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d80000ca0 con 0x7f0d8c03a850 2024-09-10T18:34:31.876 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.874+0000 7f0d977fe640 1 -- 172.21.15.175:0/3894973371 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f0d80000ca0 con 0x7f0d8c03a850 2024-09-10T18:34:31.876 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mon update... 2024-09-10T18:34:31.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.876+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 >> v1:172.21.15.131:6800/3123609852 conn(0x7f0d8c03a850 legacy=0x7f0d8c03cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:31.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.876+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 >> v1:172.21.15.131:6789/0 conn(0x7f0da0101000 legacy=0x7f0da010c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:31.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.876+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 shutdown_connections 2024-09-10T18:34:31.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.876+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 >> 172.21.15.175:0/3894973371 conn(0x7f0da00f6bf0 msgr2=0x7f0da0101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:31.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.876+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 shutdown_connections 2024-09-10T18:34:31.878 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:31.876+0000 7f0da4af2640 1 -- 172.21.15.175:0/3894973371 wait complete. 2024-09-10T18:34:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:31 smithi131 ceph-mon[29369]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:32.462 DEBUG:teuthology.orchestra.run.smithi131:mon.c> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:34:32.465 DEBUG:teuthology.orchestra.run.smithi175:mon.b> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:34:32.468 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:34:32.468 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:34:33.267 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='client.14182 v1:172.21.15.175:0/3894973371' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: Saving service mon spec with placement smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b;count:3 2024-09-10T18:34:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:32 smithi131 ceph-mon[29369]: Deploying daemon mon.b on smithi175 2024-09-10T18:34:34.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:33 smithi131 ceph-mon[29369]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.319+0000 7f96d5381640 1 -- 172.21.15.175:0/2541751968 >> v1:172.21.15.131:6789/0 conn(0x7f96d0102100 legacy=0x7f96d01024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.319+0000 7f96d5381640 1 -- 172.21.15.175:0/2541751968 shutdown_connections 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.319+0000 7f96d5381640 1 -- 172.21.15.175:0/2541751968 >> 172.21.15.175:0/2541751968 conn(0x7f96d00f7cf0 msgr2=0x7f96d00fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.319+0000 7f96d5381640 1 -- 172.21.15.175:0/2541751968 shutdown_connections 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.319+0000 7f96d5381640 1 -- 172.21.15.175:0/2541751968 wait complete. 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.319+0000 7f96d5381640 1 Processor -- start 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96d5381640 1 -- start start 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96d5381640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f96d0111da0 con 0x7f96d0102100 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96ceffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f96d0102100 0x7f96d0111690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.175:55756/0 (socket says 172.21.15.175:55756) 2024-09-10T18:34:35.336 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96ceffd640 1 -- 172.21.15.175:0/877979882 learned_addr learned my addr 172.21.15.175:0/877979882 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524101033 0 0) 0x7f96d0111da0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96bc0053f0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.320+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1115260404 0 0) 0x7f96bc0053f0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f96d0111da0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f96c0002020 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3630496429 0 0) 0x7f96d0111da0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d0111f30 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d0112200 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d0114720 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f96c00015d0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.321+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 944678457 0 0) 0x7f96c00035d0 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.322+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f96c0003810 con 0x7f96d0102100 2024-09-10T18:34:35.337 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.322+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b0005010 con 0x7f96d0102100 2024-09-10T18:34:35.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.322+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7f96c0047d40 con 0x7f96d0102100 2024-09-10T18:34:35.338 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.327+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f96c0016ae0 con 0x7f96d0102100 2024-09-10T18:34:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.549+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f96b00052c0 con 0x7f96d0102100 2024-09-10T18:34:35.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.549+0000 7f96c77fe640 1 -- 172.21.15.175:0/877979882 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+683 (unknown 2237029548 0 1014418625) 0x7f96c00164e0 con 0x7f96d0102100 2024-09-10T18:34:35.551 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:34:35.552 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":1,"fsid":"0846fc60-6fa3-11ef-bcea-c7b262605968","modified":"2024-09-10T18:33:05.034290Z","created":"2024-09-10T18:33:05.034290Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:34:35.552 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 1 2024-09-10T18:34:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.551+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 >> v1:172.21.15.131:6800/3123609852 conn(0x7f96bc03a7a0 legacy=0x7f96bc03cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.551+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 >> v1:172.21.15.131:6789/0 conn(0x7f96d0102100 legacy=0x7f96d0111690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:35.553 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.552+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 shutdown_connections 2024-09-10T18:34:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.552+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 >> 172.21.15.175:0/877979882 conn(0x7f96d00f7cf0 msgr2=0x7f96d0102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.552+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 shutdown_connections 2024-09-10T18:34:35.554 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:35.552+0000 7f96d5381640 1 -- 172.21.15.175:0/877979882 wait complete. 2024-09-10T18:34:36.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:35 smithi131 ceph-mon[29369]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:36.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/877979882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:37.333 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:34:37.333 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:34:37.616 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:38.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:37 smithi131 ceph-mon[29369]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:38.788 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:38 smithi175 podman[33295]: 2024-09-10 18:34:38.666649038 +0000 UTC m=+0.764710235 container create 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:34:39.117 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 podman[33295]: 2024-09-10 18:34:39.07122734 +0000 UTC m=+1.169288540 container init 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 podman[33295]: 2024-09-10 18:34:39.07628573 +0000 UTC m=+1.174346927 container start 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: pidfile_write: ignore empty --pid-file 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: load: jerasure load: lrc 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:34:39.118 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: DB SUMMARY 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: DB Session ID: TAOFCEYTBGTHHJA2IWSN 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 476 ; 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:34:39.119 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.env: 0x55b7149cb440 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.fs: Posix File System 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.info_log: 0x55b715328e20 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.statistics: (nil) 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.use_fsync: 0 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:34:39.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.db_log_dir: 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:34:39.121 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.write_buffer_manager: 0x55b7153e67e0 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:34:39.122 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.unordered_write: 0 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:34:39.123 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.row_cache: None 2024-09-10T18:34:39.124 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.wal_filter: None 2024-09-10T18:34:39.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:34:39.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:34:39.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:34:39.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:34:39.125 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:34:39.126 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:34:39.127 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_open_files: -1 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Compression algorithms supported: 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kZSTD supported: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:34:39.128 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.merge_operator: 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_filter: None 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:34:39.129 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b7152cfa38) 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_type: 0 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: data_block_index_type: 0 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_shortening: 1 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:34:39.130 INFO:journalctl@ceph.mon.b.smithi175.stdout: hash_index_allow_collision: 1 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: checksum: 1 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: no_block_cache: 0 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache: 0x55b7153051f0 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_options: 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: capacity : 536870912 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: num_shard_bits : 4 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: strict_capacity_limit : 0 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_cache_compressed: (nil) 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: persistent_cache: (nil) 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_size: 4096 2024-09-10T18:34:39.131 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_size_deviation: 10 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_restart_interval: 16 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: index_block_restart_interval: 1 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: metadata_block_size: 4096 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: partition_filters: 0 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: use_delta_encoding: 1 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: whole_key_filtering: 1 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: verify_compression: 0 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: format_version: 4 2024-09-10T18:34:39.132 INFO:journalctl@ceph.mon.b.smithi175.stdout: enable_index_compression: 1 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout: block_align: 0 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression: NoCompression 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.num_levels: 7 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:34:39.133 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:34:39.134 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:34:39.135 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:34:39.136 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:34:39.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.ttl: 2592000 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:34:39.138 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993279100233, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993279100524, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1536, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 488, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993279, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "8ceadd57-d618-49d5-9714-bd9db8350f0b", "db_session_id": "TAOFCEYTBGTHHJA2IWSN"}} 2024-09-10T18:34:39.139 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993279100827, "job": 1, "event": "recovery_finished"} 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55b715338700 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: DB pointer 0x55b715308000 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** DB Stats ** 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:34:39.140 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: L0 1/0 1.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: Sum 1/0 1.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:39.141 INFO:journalctl@ceph.mon.b.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:39.142 INFO:journalctl@ceph.mon.b.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: L0 1/0 1.50 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: Sum 1/0 1.50 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.143 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:39.144 INFO:journalctl@ceph.mon.b.smithi175.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: using public_addrv v1:172.21.15.175:6789/0 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: starting mon.b rank -1 at public addrs v1:172.21.15.175:6789/0 at bind addrs v1:172.21.15.175:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(???) e0 preinit fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout: e1 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:34:39.145 INFO:journalctl@ceph.mon.b.smithi175.stdout: legacy client fscid: -1 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout: 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout: No filesystems configured 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:39.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2590465671' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='client.14182 v1:172.21.15.175:0/3894973371' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: Saving service mon spec with placement smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b;count:3 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:39.147 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: Deploying daemon mon.b on smithi175 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/877979882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:39.148 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 ceph-mon[33399]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:34:39.381 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 bash[33295]: 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 2024-09-10T18:34:39.381 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:39 smithi175 systemd[1]: Started Ceph mon.b for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:34:44.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.136+0000 7f18d7b09640 1 -- 172.21.15.175:0/2574719613 >> v1:172.21.15.131:6789/0 conn(0x7f18bc003520 legacy=0x7f18bc0059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.136+0000 7f18d7b09640 1 -- 172.21.15.175:0/2574719613 shutdown_connections 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.136+0000 7f18d7b09640 1 -- 172.21.15.175:0/2574719613 >> 172.21.15.175:0/2574719613 conn(0x7f18d00f7e10 msgr2=0x7f18d00fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.137+0000 7f18d7b09640 1 -- 172.21.15.175:0/2574719613 shutdown_connections 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.137+0000 7f18d7b09640 1 -- 172.21.15.175:0/2574719613 wait complete. 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.137+0000 7f18d7b09640 1 Processor -- start 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.137+0000 7f18d7b09640 1 -- start start 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.138+0000 7f18d7b09640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f18d0117d10 con 0x7f18bc003520 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.138+0000 7f18d7b09640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f18d0118eb0 con 0x7f18d0111ef0 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.138+0000 7f18d50a4640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f18d0111ef0 0x7f18d01165f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:56384/0 (socket says 172.21.15.175:56384) 2024-09-10T18:34:44.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.138+0000 7f18d50a4640 1 -- 172.21.15.175:0/645923113 learned_addr learned my addr 172.21.15.175:0/645923113 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:34:44.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.139+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 488615469 0 0) 0x7f18d0117d10 con 0x7f18bc003520 2024-09-10T18:34:44.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.139+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b8005540 con 0x7f18bc003520 2024-09-10T18:34:44.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.139+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2615383407 0 0) 0x7f18b8005540 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.139+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f18d0117d10 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.139+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 1578365375 0 0) 0x7f18c0002ef0 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3124806825 0 0) 0x7f18d0117d10 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 >> v1:172.21.15.175:6789/0 conn(0x7f18d0111ef0 legacy=0x7f18d01165f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d011a050 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0117ea0 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d01183b0 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f18c0001ea0 con 0x7f18bc003520 2024-09-10T18:34:44.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.140+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 1578365375 0 0) 0x7f18c0003810 con 0x7f18bc003520 2024-09-10T18:34:44.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.141+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f18c0003970 con 0x7f18bc003520 2024-09-10T18:34:44.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.141+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d00fd9b0 con 0x7f18bc003520 2024-09-10T18:34:44.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.141+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7f18c0045270 con 0x7f18bc003520 2024-09-10T18:34:44.147 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.146+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f18c0014310 con 0x7f18bc003520 2024-09-10T18:34:44.375 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.373+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f18d005ed70 con 0x7f18bc003520 2024-09-10T18:34:44.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.374+0000 7f18ce7fc640 1 -- 172.21.15.175:0/645923113 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+901 (unknown 3557084514 0 1479606141) 0x7f18c0013d10 con 0x7f18bc003520 2024-09-10T18:34:44.376 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:34:44.376 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":2,"fsid":"0846fc60-6fa3-11ef-bcea-c7b262605968","modified":"2024-09-10T18:34:39.120922Z","created":"2024-09-10T18:33:05.034290Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:34:44.377 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 2 2024-09-10T18:34:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.376+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 >> v1:172.21.15.131:6800/3123609852 conn(0x7f18b803aa30 legacy=0x7f18b803cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.376+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 >> v1:172.21.15.131:6789/0 conn(0x7f18bc003520 legacy=0x7f18d01117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.376+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 shutdown_connections 2024-09-10T18:34:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.376+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 >> 172.21.15.175:0/645923113 conn(0x7f18d00f7e10 msgr2=0x7f18d00f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.376+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 shutdown_connections 2024-09-10T18:34:44.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:44.376+0000 7f18d7b09640 1 -- 172.21.15.175:0/645923113 wait complete. 2024-09-10T18:34:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: mon.a calling monitor election 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: mon.b calling monitor election 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: monmap e2: 2 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: fsmap 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: mgrmap e13: y(active, since 41s) 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: overall HEALTH_OK 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:44.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:44 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: mon.a calling monitor election 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: mon.b calling monitor election 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: monmap e2: 2 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: fsmap 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: mgrmap e13: y(active, since 41s) 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: overall HEALTH_OK 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:44.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:44 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:45.475 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:45 smithi131 ceph-mon[29369]: Deploying daemon mon.c on smithi131 2024-09-10T18:34:45.475 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:45 smithi131 ceph-mon[29369]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:45.475 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/645923113' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:45.475 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:45 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:45.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:45 smithi175 ceph-mon[33399]: Deploying daemon mon.c on smithi131 2024-09-10T18:34:45.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:45 smithi175 ceph-mon[33399]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:45.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/645923113' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:45.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:45 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:45.962 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:34:45.962 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:34:46.222 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:47.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 podman[40848]: 2024-09-10 18:34:47.20771696 +0000 UTC m=+0.126794043 container create e89026a64335aed583dcd9035af67e5db79a2f30050eba7d5ecf2ee40dbade3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T18:34:47.547 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:47 smithi175 ceph-mon[33399]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[29369]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 podman[40848]: 2024-09-10 18:34:47.401678228 +0000 UTC m=+0.320755312 container init e89026a64335aed583dcd9035af67e5db79a2f30050eba7d5ecf2ee40dbade3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:34:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 podman[40848]: 2024-09-10 18:34:47.40638905 +0000 UTC m=+0.325466131 container start e89026a64335aed583dcd9035af67e5db79a2f30050eba7d5ecf2ee40dbade3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T18:34:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:34:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pidfile_write: ignore empty --pid-file 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: load: jerasure load: lrc 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: DB SUMMARY 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: DB Session ID: QTJD7BLWF5J1Y197YNP3 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T18:34:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.env: 0x55848eb94440 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.fs: Posix File System 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.info_log: 0x55849024ae80 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:34:47.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.statistics: (nil) 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.use_fsync: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:34:47.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.db_log_dir: 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.write_buffer_manager: 0x558490308840 2024-09-10T18:34:47.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.unordered_write: 0 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:34:47.833 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.row_cache: None 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.wal_filter: None 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:34:47.834 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:34:47.835 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_open_files: -1 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Compression algorithms supported: 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kZSTD supported: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:34:47.836 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.merge_operator: 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_filter: None 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:34:47.837 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5584901f1a38) 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_type: 0 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_index_type: 0 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_shortening: 1 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: hash_index_allow_collision: 1 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: checksum: 1 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: no_block_cache: 0 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache: 0x558490227350 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_options: 2024-09-10T18:34:47.838 INFO:journalctl@ceph.mon.c.smithi131.stdout: capacity : 536870912 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: num_shard_bits : 4 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: strict_capacity_limit : 0 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_cache_compressed: (nil) 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: persistent_cache: (nil) 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size: 4096 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_size_deviation: 10 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_restart_interval: 16 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: index_block_restart_interval: 1 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: metadata_block_size: 4096 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: partition_filters: 0 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: use_delta_encoding: 1 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: whole_key_filtering: 1 2024-09-10T18:34:47.839 INFO:journalctl@ceph.mon.c.smithi131.stdout: verify_compression: 0 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout: format_version: 4 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_index_compression: 1 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout: block_align: 0 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression: NoCompression 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.num_levels: 7 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:34:47.840 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:34:47.841 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:34:47.842 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:34:47.843 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:34:47.844 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.ttl: 2592000 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:34:47.845 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993287430500, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993287430811, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1626, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993287, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f42ff4ba-217f-4c02-bc65-ab8fea892591", "db_session_id": "QTJD7BLWF5J1Y197YNP3"}} 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993287431123, "job": 1, "event": "recovery_finished"} 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55849025a700 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: DB pointer 0x55849022a000 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** DB Stats ** 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:34:47.846 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:47.847 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.848 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: L0 1/0 1.59 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: Sum 1/0 1.59 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:47.849 INFO:journalctl@ceph.mon.c.smithi131.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: using public_addrv v1:172.21.15.131:6790/0 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: starting mon.c rank -1 at public addrs v1:172.21.15.131:6790/0 at bind addrs v1:172.21.15.131:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(???) e0 preinit fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:47.850 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout: e1 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout: legacy client fscid: -1 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout: 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout: No filesystems configured 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:34:47.851 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2590465671' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='client.14182 v1:172.21.15.175:0/3894973371' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: Saving service mon spec with placement smithi131:[v1:172.21.15.131:6789]=a;smithi131:[v1:172.21.15.131:6790]=c;smithi175:[v1:172.21.15.175:6789]=b;count:3 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:47.852 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: Deploying daemon mon.b on smithi175 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/877979882' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.a calling monitor election 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.853 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.b calling monitor election 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.854 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: monmap e2: 2 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: fsmap 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mgrmap e13: y(active, since 41s) 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: overall HEALTH_OK 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: Deploying daemon mon.c on smithi131 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.855 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/645923113' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:47.856 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:47.856 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:47.856 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 ceph-mon[40885]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-09-10T18:34:47.856 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 bash[40848]: e89026a64335aed583dcd9035af67e5db79a2f30050eba7d5ecf2ee40dbade3d 2024-09-10T18:34:47.856 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:47 smithi131 systemd[1]: Started Ceph mon.c for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:34:52.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.469+0000 7f23691a0640 1 -- 172.21.15.175:0/1611677979 >> v1:172.21.15.131:6789/0 conn(0x7f2344003520 legacy=0x7f23440059a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:52.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.469+0000 7f23691a0640 1 -- 172.21.15.175:0/1611677979 shutdown_connections 2024-09-10T18:34:52.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.469+0000 7f23691a0640 1 -- 172.21.15.175:0/1611677979 >> 172.21.15.175:0/1611677979 conn(0x7f2364071150 msgr2=0x7f2364071550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:52.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.469+0000 7f23691a0640 1 -- 172.21.15.175:0/1611677979 shutdown_connections 2024-09-10T18:34:52.471 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.469+0000 7f23691a0640 1 -- 172.21.15.175:0/1611677979 wait complete. 2024-09-10T18:34:52.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f23691a0640 1 Processor -- start 2024-09-10T18:34:52.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f23691a0640 1 -- start start 2024-09-10T18:34:52.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f23691a0640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f236411bb70 con 0x7f2364118070 2024-09-10T18:34:52.472 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f23691a0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f236411cd10 con 0x7f2344003520 2024-09-10T18:34:52.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f23691a0640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f236411def0 con 0x7f236410cfe0 2024-09-10T18:34:52.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f2362d9d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2344003520 0x7f236410c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:56396/0 (socket says 172.21.15.175:56396) 2024-09-10T18:34:52.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.470+0000 7f2362d9d640 1 -- 172.21.15.175:0/3737668278 learned_addr learned my addr 172.21.15.175:0/3737668278 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.471+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582482289 0 0) 0x7f236411bb70 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.471+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2350003b80 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.471+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1520980756 0 0) 0x7f2350003b80 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f236411bb70 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f235c001b40 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 405724664 0 0) 0x7f236411bb70 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 >> v1:172.21.15.131:6790/0 conn(0x7f236410cfe0 legacy=0x7f2364116950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 >> v1:172.21.15.175:6789/0 conn(0x7f2344003520 legacy=0x7f236410c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236411f110 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236411bd60 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f236411c1f0 con 0x7f2364118070 2024-09-10T18:34:52.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f235c0021b0 con 0x7f2364118070 2024-09-10T18:34:52.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.472+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f235c003440 con 0x7f2364118070 2024-09-10T18:34:52.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.473+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f235c0035c0 con 0x7f2364118070 2024-09-10T18:34:52.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.473+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7f235c045e60 con 0x7f2364118070 2024-09-10T18:34:52.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.473+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23440069d0 con 0x7f2364118070 2024-09-10T18:34:52.479 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.478+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f235c013cb0 con 0x7f2364118070 2024-09-10T18:34:52.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.709+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2344006c80 con 0x7f2364118070 2024-09-10T18:34:52.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.710+0000 7f235b7fe640 1 -- 172.21.15.175:0/3737668278 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1117 (unknown 409899127 0 918773119) 0x7f235c013ad0 con 0x7f2364118070 2024-09-10T18:34:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-10T18:34:52.712 INFO:teuthology.orchestra.run.smithi175.stdout:{"epoch":3,"fsid":"0846fc60-6fa3-11ef-bcea-c7b262605968","modified":"2024-09-10T18:34:47.459086Z","created":"2024-09-10T18:33:05.034290Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6789","nonce":0}]},"addr":"172.21.15.131:6789/0","public_addr":"172.21.15.131:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6789","nonce":0}]},"addr":"172.21.15.175:6789/0","public_addr":"172.21.15.175:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6790","nonce":0}]},"addr":"172.21.15.131:6790/0","public_addr":"172.21.15.131:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:34:52.712 INFO:teuthology.orchestra.run.smithi175.stderr:dumped monmap epoch 3 2024-09-10T18:34:52.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.712+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 >> v1:172.21.15.131:6800/3123609852 conn(0x7f235003a710 legacy=0x7f235003cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:52.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.712+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 >> v1:172.21.15.131:6789/0 conn(0x7f2364118070 legacy=0x7f236411a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:52.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.712+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 shutdown_connections 2024-09-10T18:34:52.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.712+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 >> 172.21.15.175:0/3737668278 conn(0x7f2364071150 msgr2=0x7f23641005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:52.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.712+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 shutdown_connections 2024-09-10T18:34:52.714 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:34:52.712+0000 7f23691a0640 1 -- 172.21.15.175:0/3737668278 wait complete. 2024-09-10T18:34:52.723 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:52.723 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: mon.a calling monitor election 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: mon.b calling monitor election 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:52.724 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: fsmap 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: mgrmap e13: y(active, since 50s) 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: mon.c (rank 2) addr v1:172.21.15.131:6790/0 is down (out of quorum) 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.725 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: mon.a calling monitor election 2024-09-10T18:34:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: mon.b calling monitor election 2024-09-10T18:34:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: fsmap 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: mgrmap e13: y(active, since 50s) 2024-09-10T18:34:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: mon.c (rank 2) addr v1:172.21.15.131:6790/0 is down (out of quorum) 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:52.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:53.280 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:34:53.280 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:34:53.656 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:34:53.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:53 smithi131 ceph-mon[29369]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:53.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:53 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:53.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/3737668278' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:53.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:53 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:53.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:53 smithi175 ceph-mon[33399]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:53.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:53 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:53.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/3737668278' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:53.844 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:53 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: mon.a calling monitor election 2024-09-10T18:34:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: mon.b calling monitor election 2024-09-10T18:34:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: fsmap 2024-09-10T18:34:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: mgrmap e13: y(active, since 50s) 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: mon.c (rank 2) addr v1:172.21.15.131:6790/0 is down (out of quorum) 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:54.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/3737668278' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:34:54.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:54 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: mon.c calling monitor election 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: mon.c calling monitor election 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: mon.b calling monitor election 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: mon.a calling monitor election 2024-09-10T18:34:55.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: fsmap 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: mgrmap e13: y(active, since 52s) 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Cluster is now healthy 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: overall HEALTH_OK 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:34:55.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:55.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:55.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:55.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:55 smithi175 ceph-mon[33399]: Reconfiguring daemon mon.a on smithi131 2024-09-10T18:34:55.215 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: mon.c calling monitor election 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: mon.c calling monitor election 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: mon.b calling monitor election 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: mon.a calling monitor election 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:34:55.216 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: fsmap 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: mgrmap e13: y(active, since 52s) 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Cluster is now healthy 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: overall HEALTH_OK 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.217 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[29369]: Reconfiguring daemon mon.a on smithi131 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: mon.c calling monitor election 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:55.218 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: mon.c calling monitor election 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: mon.b calling monitor election 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: mon.a calling monitor election 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: monmap e3: 3 mons at {a=v1:172.21.15.131:6789/0,b=v1:172.21.15.175:6789/0,c=v1:172.21.15.131:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: fsmap 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: mgrmap e13: y(active, since 52s) 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Cluster is now healthy 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: overall HEALTH_OK 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.219 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:55.220 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:55 smithi131 ceph-mon[40885]: Reconfiguring daemon mon.a on smithi131 2024-09-10T18:34:56.096 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.088+0000 7f43d2bcb640 1 -- 172.21.15.131:0/3883800668 >> v1:172.21.15.131:6789/0 conn(0x7f43cc102100 legacy=0x7f43cc1024d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.089+0000 7f43d2bcb640 1 -- 172.21.15.131:0/3883800668 shutdown_connections 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.089+0000 7f43d2bcb640 1 -- 172.21.15.131:0/3883800668 >> 172.21.15.131:0/3883800668 conn(0x7f43cc0f7cd0 msgr2=0x7f43cc0fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.089+0000 7f43d2bcb640 1 -- 172.21.15.131:0/3883800668 shutdown_connections 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.089+0000 7f43d2bcb640 1 -- 172.21.15.131:0/3883800668 wait complete. 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.089+0000 7f43d2bcb640 1 Processor -- start 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.089+0000 7f43d2bcb640 1 -- start start 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43d2bcb640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f43cc11baf0 con 0x7f43cc102100 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43d2bcb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f43cc11ccb0 con 0x7f43cc117e30 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43d2bcb640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f43cc11deb0 con 0x7f43cc112010 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43d0967640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f43cc102100 0x7f43cc111900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51018/0 (socket says 172.21.15.131:51018) 2024-09-10T18:34:56.097 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43d0967640 1 -- 172.21.15.131:0/1709800753 learned_addr learned my addr 172.21.15.131:0/1709800753 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161138176 0 0) 0x7f43cc11baf0 con 0x7f43cc102100 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43bc0034e0 con 0x7f43cc102100 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890456132 0 0) 0x7f43cc11ccb0 con 0x7f43cc117e30 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.090+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43cc11baf0 con 0x7f43cc117e30 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 168102480 0 0) 0x7f43cc11baf0 con 0x7f43cc117e30 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43cc11ccb0 con 0x7f43cc117e30 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f43c4001b40 con 0x7f43cc117e30 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1791695388 0 0) 0x7f43bc0034e0 con 0x7f43cc102100 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f43cc11baf0 con 0x7f43cc102100 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f43c00010a0 con 0x7f43cc102100 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1480301548 0 0) 0x7f43cc11baf0 con 0x7f43cc102100 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 >> v1:172.21.15.131:6790/0 conn(0x7f43cc112010 legacy=0x7f43cc116710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:56.098 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 >> v1:172.21.15.175:6789/0 conn(0x7f43cc117e30 legacy=0x7f43cc11a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc11f0b0 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43cc11e0a0 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.091+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f43cc11e5b0 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.092+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43cc0fd870 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.093+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f43c00022e0 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.093+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f43c0003540 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.093+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f43c0003740 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.094+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7f43c0047b10 con 0x7f43cc102100 2024-09-10T18:34:56.099 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.098+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f43c0016a00 con 0x7f43cc102100 2024-09-10T18:34:56.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.260+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f43cc10d010 con 0x7f43cc102100 2024-09-10T18:34:56.262 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.260+0000 7f43c97fa640 1 -- 172.21.15.131:0/1709800753 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+193 (unknown 2189217731 0 3131512601) 0x7f43c0016400 con 0x7f43cc102100 2024-09-10T18:34:56.262 INFO:teuthology.orchestra.run.smithi131.stdout:# minimal ceph.conf for 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:56.263 INFO:teuthology.orchestra.run.smithi131.stdout:[global] 2024-09-10T18:34:56.263 INFO:teuthology.orchestra.run.smithi131.stdout: fsid = 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:34:56.263 INFO:teuthology.orchestra.run.smithi131.stdout: mon_host = 172.21.15.131:6789/0 172.21.15.175:6789/0 v1:172.21.15.131:6790/0 2024-09-10T18:34:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.262+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 >> v1:172.21.15.131:6800/3123609852 conn(0x7f43bc03a780 legacy=0x7f43bc03cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.262+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 >> v1:172.21.15.131:6789/0 conn(0x7f43cc102100 legacy=0x7f43cc111900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:34:56.264 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.262+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 shutdown_connections 2024-09-10T18:34:56.264 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:56 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:56.265 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:56 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.262+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 >> 172.21.15.131:0/1709800753 conn(0x7f43cc0f7cd0 msgr2=0x7f43cc1038b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:34:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.262+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 shutdown_connections 2024-09-10T18:34:56.265 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:34:56.262+0000 7f43d2bcb640 1 -- 172.21.15.131:0/1709800753 wait complete. 2024-09-10T18:34:56.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:56 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:34:57.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1709800753' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:57.204 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[29369]: Reconfiguring daemon mon.c on smithi131 2024-09-10T18:34:57.205 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1709800753' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:57.206 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:57 smithi131 ceph-mon[40885]: Reconfiguring daemon mon.c on smithi131 2024-09-10T18:34:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1709800753' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:34:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:57 smithi175 ceph-mon[33399]: Reconfiguring daemon mon.c on smithi131 2024-09-10T18:34:57.512 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:34:57.512 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:34:57.512 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:34:57.539 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:34:57.539 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:57.604 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:34:57.604 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:34:57.637 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:34:57.637 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:57.706 INFO:tasks.cephadm:Adding mgr.y on smithi131 2024-09-10T18:34:57.707 INFO:tasks.cephadm:Adding mgr.x on smithi175 2024-09-10T18:34:57.707 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi131=y;smithi175=x' 2024-09-10T18:34:57.988 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:34:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:58.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:58.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:58.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:58.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:34:58.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:34:58.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:34:58.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:34:59.558 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:59 smithi131 ceph-mon[40885]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:34:59.558 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:59 smithi131 ceph-mon[40885]: Reconfiguring daemon mon.b on smithi175 2024-09-10T18:34:59.558 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:34:59 smithi131 ceph-mon[40885]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:59.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:59 smithi131 ceph-mon[29369]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:34:59.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:59 smithi131 ceph-mon[29369]: Reconfiguring daemon mon.b on smithi175 2024-09-10T18:34:59.558 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:34:59 smithi131 ceph-mon[29369]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:34:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:59 smithi175 ceph-mon[33399]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:34:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:59 smithi175 ceph-mon[33399]: Reconfiguring daemon mon.b on smithi175 2024-09-10T18:34:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:34:59 smithi175 ceph-mon[33399]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:00.415 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- 172.21.15.175:0/1980512446 >> v1:172.21.15.175:6789/0 conn(0x7f9a980ff950 legacy=0x7f9a98101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- 172.21.15.175:0/1980512446 shutdown_connections 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- 172.21.15.175:0/1980512446 >> 172.21.15.175:0/1980512446 conn(0x7f9a980f6c10 msgr2=0x7f9a980f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- 172.21.15.175:0/1980512446 shutdown_connections 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- 172.21.15.175:0/1980512446 wait complete. 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 Processor -- start 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- start start 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a98117740 con 0x7f9a980ff950 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a98118940 con 0x7f9a980fb450 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9f798640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a98119b40 con 0x7f9a98103570 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9d534640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f9a980fb450 0x7f9a98066530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:42840/0 (socket says 172.21.15.175:42840) 2024-09-10T18:35:00.416 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.410+0000 7f9a9d534640 1 -- 172.21.15.175:0/772863693 learned_addr learned my addr 172.21.15.175:0/772863693 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1964514452 0 0) 0x7f9a98118940 con 0x7f9a980fb450 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a840034e0 con 0x7f9a980fb450 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654671510 0 0) 0x7f9a98119b40 con 0x7f9a98103570 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a98118940 con 0x7f9a98103570 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516452629 0 0) 0x7f9a98117740 con 0x7f9a980ff950 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a98119b40 con 0x7f9a980ff950 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2866486907 0 0) 0x7f9a840034e0 con 0x7f9a980fb450 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a98117740 con 0x7f9a980fb450 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3188698077 0 0) 0x7f9a98118940 con 0x7f9a98103570 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a840034e0 con 0x7f9a98103570 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a94002970 con 0x7f9a980fb450 2024-09-10T18:35:00.417 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a90001c40 con 0x7f9a98103570 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 832063631 0 0) 0x7f9a98119b40 con 0x7f9a980ff950 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a98118940 con 0x7f9a980ff950 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a88001880 con 0x7f9a980ff950 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3640170520 0 0) 0x7f9a98117740 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 >> v1:172.21.15.131:6790/0 conn(0x7f9a98103570 legacy=0x7f9a98115e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 >> v1:172.21.15.131:6789/0 conn(0x7f9a980ff950 legacy=0x7f9a98112490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9811ad40 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a98119d30 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f9a94002ba0 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.411+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a94005af0 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.412+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9811a200 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.412+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7f9a94010f80 con 0x7f9a980fb450 2024-09-10T18:35:00.418 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.412+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7f9a94047850 con 0x7f9a980fb450 2024-09-10T18:35:00.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.412+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9804bf60 con 0x7f9a980fb450 2024-09-10T18:35:00.419 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.417+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f9a94012d80 con 0x7f9a980fb450 2024-09-10T18:35:00.581 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.579+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi175=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a980feac0 con 0x7f9a8403a840 2024-09-10T18:35:00.589 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.587+0000 7f9a8dffb640 1 -- 172.21.15.175:0/772863693 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f9a980feac0 con 0x7f9a8403a840 2024-09-10T18:35:00.589 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled mgr update... 2024-09-10T18:35:00.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.589+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 >> v1:172.21.15.131:6800/3123609852 conn(0x7f9a8403a840 legacy=0x7f9a8403ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:00.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.589+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 >> v1:172.21.15.175:6789/0 conn(0x7f9a980fb450 legacy=0x7f9a98066530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:00.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.589+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 shutdown_connections 2024-09-10T18:35:00.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.589+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 >> 172.21.15.175:0/772863693 conn(0x7f9a980f6c10 msgr2=0x7f9a98105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:00.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.590+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 shutdown_connections 2024-09-10T18:35:00.592 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:35:00.590+0000 7f9a9f798640 1 -- 172.21.15.175:0/772863693 wait complete. 2024-09-10T18:35:01.177 DEBUG:teuthology.orchestra.run.smithi175:mgr.x> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:35:01.179 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:35:01.179 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:35:01.179 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:35:01.196 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:35:01.196 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_1 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 18:34:46.926736182 +0000 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 18:30:15.399629203 +0000 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 18:30:15.399629203 +0000 2024-09-10T18:35:01.252 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 18:30:15.399629203 +0000 2024-09-10T18:35:01.253 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:35:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T18:35:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T18:35:01.317 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000276039 s, 1.9 MB/s 2024-09-10T18:35:01.318 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:35:01.375 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_2 2024-09-10T18:35:01.432 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:35:01.432 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:01.432 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-10T18:35:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 18:34:46.926736182 +0000 2024-09-10T18:35:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 18:30:15.402628885 +0000 2024-09-10T18:35:01.433 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 18:30:15.402628885 +0000 2024-09-10T18:35:01.433 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 18:30:15.402628885 +0000 2024-09-10T18:35:01.433 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:01.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='client.24106 v1:172.21.15.175:0/772863693' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: Saving service mgr spec with placement smithi131=y;smithi175=x;count:2 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[29369]: Deploying daemon mgr.x on smithi175 2024-09-10T18:35:01.495 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='client.24106 v1:172.21.15.175:0/772863693' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: Saving service mgr spec with placement smithi131=y;smithi175=x;count:2 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:01.496 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:35:01.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:35:01.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:35:01.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.497 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:01 smithi131 ceph-mon[40885]: Deploying daemon mgr.x on smithi175 2024-09-10T18:35:01.498 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T18:35:01.498 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T18:35:01.498 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000215068 s, 2.4 MB/s 2024-09-10T18:35:01.498 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:35:01.554 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_3 2024-09-10T18:35:01.563 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.563 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.563 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='client.24106 v1:172.21.15.175:0/772863693' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi131=y;smithi175=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: Saving service mgr spec with placement smithi131=y;smithi175=x;count:2 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:01.564 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:01.565 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:35:01.565 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:35:01.565 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:35:01.565 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:01.565 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:01 smithi175 ceph-mon[33399]: Deploying daemon mgr.x on smithi175 2024-09-10T18:35:01.608 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:35:01.608 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 18:34:46.927736161 +0000 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 18:30:15.400629097 +0000 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 18:30:15.400629097 +0000 2024-09-10T18:35:01.609 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 18:30:15.400629097 +0000 2024-09-10T18:35:01.609 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:35:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T18:35:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T18:35:01.672 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000201497 s, 2.5 MB/s 2024-09-10T18:35:01.673 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:35:01.730 DEBUG:teuthology.orchestra.run.smithi131:> stat /dev/vg_nvme/lv_4 2024-09-10T18:35:01.786 INFO:teuthology.orchestra.run.smithi131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:35:01.786 INFO:teuthology.orchestra.run.smithi131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:01.786 INFO:teuthology.orchestra.run.smithi131.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-10T18:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout:Access: 2024-09-10 18:34:46.927736161 +0000 2024-09-10T18:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout:Modify: 2024-09-10 18:30:15.400629097 +0000 2024-09-10T18:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout:Change: 2024-09-10 18:30:15.400629097 +0000 2024-09-10T18:35:01.787 INFO:teuthology.orchestra.run.smithi131.stdout: Birth: 2024-09-10 18:30:15.400629097 +0000 2024-09-10T18:35:01.787 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:35:01.850 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records in 2024-09-10T18:35:01.850 INFO:teuthology.orchestra.run.smithi131.stderr:1+0 records out 2024-09-10T18:35:01.850 INFO:teuthology.orchestra.run.smithi131.stderr:512 bytes copied, 0.000202539 s, 2.5 MB/s 2024-09-10T18:35:01.851 DEBUG:teuthology.orchestra.run.smithi131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:35:01.908 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:35:01.908 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:35:01.925 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:35:01.925 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-09-10T18:35:01.981 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-10 18:34:37.527449716 +0000 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-10 18:30:15.414428465 +0000 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-10 18:30:15.414428465 +0000 2024-09-10T18:35:01.982 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-10 18:30:15.414428465 +0000 2024-09-10T18:35:01.982 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:35:02.045 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-10T18:35:02.046 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-10T18:35:02.046 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000269072 s, 1.9 MB/s 2024-09-10T18:35:02.047 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:35:02.103 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-10 18:34:37.528449700 +0000 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-10 18:30:15.415428645 +0000 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-10 18:30:15.415428645 +0000 2024-09-10T18:35:02.159 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-10 18:30:15.415428645 +0000 2024-09-10T18:35:02.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:35:02.222 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-10T18:35:02.222 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-10T18:35:02.223 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000199511 s, 2.6 MB/s 2024-09-10T18:35:02.224 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:35:02.280 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 976 Links: 1 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-10 18:34:37.528449700 +0000 2024-09-10T18:35:02.337 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-10 18:30:15.416428825 +0000 2024-09-10T18:35:02.338 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-10 18:30:15.416428825 +0000 2024-09-10T18:35:02.338 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-10 18:30:15.416428825 +0000 2024-09-10T18:35:02.338 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:35:02.417 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-10T18:35:02.417 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-10T18:35:02.417 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000186334 s, 2.7 MB/s 2024-09-10T18:35:02.418 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:35:02.475 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-10 18:35:02.369059976 +0000 2024-09-10T18:35:02.531 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-10 18:30:15.417429006 +0000 2024-09-10T18:35:02.532 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-10 18:30:15.417429006 +0000 2024-09-10T18:35:02.532 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-10 18:30:15.417429006 +0000 2024-09-10T18:35:02.532 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:35:02.598 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-10T18:35:02.598 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-10T18:35:02.598 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000225046 s, 2.3 MB/s 2024-09-10T18:35:02.599 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:35:02.657 INFO:tasks.cephadm:Deploying osd.0 on smithi131 with /dev/vg_nvme/lv_4... 2024-09-10T18:35:02.657 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:35:02.677 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:02 smithi175 systemd[1]: Starting Ceph mgr.x for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:35:02.905 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:03.138 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:02 smithi175 podman[35315]: 2024-09-10 18:35:02.817671449 +0000 UTC m=+0.107509958 container create 48d6e92c930db6e25bd4c3373e3a881e5bf515d0ec089631ebafa3d57b20445a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T18:35:03.138 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:02 smithi175 podman[35315]: 2024-09-10 18:35:02.72541428 +0000 UTC m=+0.015252796 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:35:03.138 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 podman[35315]: 2024-09-10 18:35:03.028287925 +0000 UTC m=+0.318126433 container init 48d6e92c930db6e25bd4c3373e3a881e5bf515d0ec089631ebafa3d57b20445a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:35:03.138 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 podman[35315]: 2024-09-10 18:35:03.033531808 +0000 UTC m=+0.323370317 container start 48d6e92c930db6e25bd4c3373e3a881e5bf515d0ec089631ebafa3d57b20445a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:35:03.409 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 bash[35315]: 48d6e92c930db6e25bd4c3373e3a881e5bf515d0ec089631ebafa3d57b20445a 2024-09-10T18:35:03.409 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 systemd[1]: Started Ceph mgr.x for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:35:03.409 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.146+0000 7f4cfa7f0ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:35:03.409 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.211+0000 7f4cfa7f0ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:03.409 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.277+0000 7f4cfa7f0ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:35:03.409 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.407+0000 7f4cfa7f0ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[29369]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[40885]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:03.677 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.552+0000 7f4cfa7f0ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:35:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-mon[33399]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:04.116 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.825+0000 7f4cfa7f0ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:35:04.116 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:03.887+0000 7f4cfa7f0ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:35:04.116 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:04.057+0000 7f4cfa7f0ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:35:04.744 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:35:04.828 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:35:04.841 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-10T18:35:04.841 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-10T18:35:04.841 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113585 s, 923 MB/s 2024-09-10T18:35:04.842 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-10T18:35:05.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:04.911+0000 7f4cfa7f0ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:35:05.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:04.976+0000 7f4cfa7f0ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:35:05.567 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:05.568 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[29369]: Reconfiguring daemon mgr.y on smithi131 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:05.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:05 smithi131 ceph-mon[40885]: Reconfiguring daemon mgr.y on smithi131 2024-09-10T18:35:05.591 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_4 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:35:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:35:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:35:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-mon[33399]: Reconfiguring daemon mgr.y on smithi131 2024-09-10T18:35:05.967 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:05.680+0000 7f4cfa7f0ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:35:05.967 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:05.789+0000 7f4cfa7f0ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:35:05.967 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:05.907+0000 7f4cfa7f0ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:06.346 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:05.965+0000 7f4cfa7f0ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:35:06.346 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.088+0000 7f4cfa7f0ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:35:06.346 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.155+0000 7f4cfa7f0ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:35:06.346 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.212+0000 7f4cfa7f0ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:35:06.346 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.343+0000 7f4cfa7f0ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:35:06.593 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:06.609 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.406+0000 7f4cfa7f0ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:35:06.609 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.470+0000 7f4cfa7f0ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:35:06.609 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.549+0000 7f4cfa7f0ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:06.609 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.606+0000 7f4cfa7f0ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:35:06.927 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:06 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:06.663+0000 7f4cfa7f0ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:35:07.669 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: Standby manager daemon x started 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:35:07.670 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: Standby manager daemon x started 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:35:07.671 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:07 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:35:07.403+0000 7f4cfa7f0ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: Standby manager daemon x started 2024-09-10T18:35:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:35:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:35:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:35:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:07 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/3495458325' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:35:08.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 -- 172.21.15.131:0/2709508600 >> v1:172.21.15.131:6790/0 conn(0x7fb64c1046c0 legacy=0x7fb64c106b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:08.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 -- 172.21.15.131:0/2709508600 shutdown_connections 2024-09-10T18:35:08.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 -- 172.21.15.131:0/2709508600 >> 172.21.15.131:0/2709508600 conn(0x7fb64c0f63c0 msgr2=0x7fb64c0f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 -- 172.21.15.131:0/2709508600 shutdown_connections 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 -- 172.21.15.131:0/2709508600 wait complete. 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 Processor -- start 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.175+0000 7fb6520da640 1 -- start start 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb6520da640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb64c10d060 con 0x7fb64c0fdbc0 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb6520da640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb64c118860 con 0x7fb64c100bc0 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb6520da640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb64c119a00 con 0x7fb64c1046c0 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb64b7fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb64c0fdbc0 0x7fb64c10c4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54622/0 (socket says 172.21.15.131:54622) 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb64b7fe640 1 -- 172.21.15.131:0/3063165190 learned_addr learned my addr 172.21.15.131:0/3063165190 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:35:08.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb64bfff640 1 --1- 172.21.15.131:0/3063165190 >> v1:172.21.15.131:6790/0 conn(0x7fb64c1046c0 0x7fb64c116130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:53554/0 (socket says 172.21.15.131:53554) 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1605434277 0 0) 0x7fb64c10d060 con 0x7fb64c0fdbc0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb634003120 con 0x7fb64c0fdbc0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784937653 0 0) 0x7fb64c119a00 con 0x7fb64c1046c0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb64c10d060 con 0x7fb64c1046c0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307497350 0 0) 0x7fb64c118860 con 0x7fb64c100bc0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb64c119a00 con 0x7fb64c100bc0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 426015611 0 0) 0x7fb634003120 con 0x7fb64c0fdbc0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb64c118860 con 0x7fb64c0fdbc0 2024-09-10T18:35:08.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb63c0019f0 con 0x7fb64c0fdbc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 574380971 0 0) 0x7fb64c119a00 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb634003120 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb6400018c0 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2030040758 0 0) 0x7fb64c10d060 con 0x7fb64c1046c0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb64c119a00 con 0x7fb64c1046c0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb644001710 con 0x7fb64c1046c0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3676230899 0 0) 0x7fb634003120 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 >> v1:172.21.15.131:6790/0 conn(0x7fb64c1046c0 legacy=0x7fb64c116130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 >> v1:172.21.15.131:6789/0 conn(0x7fb64c0fdbc0 legacy=0x7fb64c10c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c11abe0 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.176+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c1178b0 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.177+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c117d80 con 0x7fb64c100bc0 2024-09-10T18:35:08.181 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.177+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fb6400022a0 con 0x7fb64c100bc0 2024-09-10T18:35:08.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.177+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb6400030d0 con 0x7fb64c100bc0 2024-09-10T18:35:08.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.177+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 13) v1 ==== 45514+0+0 (unknown 3155884911 0 0) 0x7fb64000e410 con 0x7fb64c100bc0 2024-09-10T18:35:08.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.177+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb638005040 con 0x7fb64c100bc0 2024-09-10T18:35:08.182 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.178+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1065+0+0 (unknown 1165621117 0 0) 0x7fb640044d20 con 0x7fb64c100bc0 2024-09-10T18:35:08.184 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.183+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb640013870 con 0x7fb64c100bc0 2024-09-10T18:35:08.347 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.345+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb638000bc0 con 0x7fb63403a800 2024-09-10T18:35:08.392 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:08.390+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mon.1 v1:172.21.15.175:6789/0 10 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 864645932 0 0) 0x7fb640043cb0 con 0x7fb64c100bc0 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[40885]: mgrmap e14: y(active, since 66s), standbys: x 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:08.747 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:08.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[29369]: mgrmap e14: y(active, since 66s), standbys: x 2024-09-10T18:35:08.748 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:08 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:35:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:08 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:08 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:08 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:08 smithi175 ceph-mon[33399]: mgrmap e14: y(active, since 66s), standbys: x 2024-09-10T18:35:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:08 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:35:09.796 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:09 smithi131 ceph-mon[29369]: from='client.24118 v1:172.21.15.131:0/3063165190' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:09.796 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:09 smithi131 ceph-mon[29369]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:09.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:09 smithi131 ceph-mon[40885]: from='client.24118 v1:172.21.15.131:0/3063165190' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:09.796 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:09 smithi131 ceph-mon[40885]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:09 smithi175 ceph-mon[33399]: from='client.24118 v1:172.21.15.131:0/3063165190' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:09 smithi175 ceph-mon[33399]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3621928698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]: dispatch 2024-09-10T18:35:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: from='client.34121 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]: dispatch 2024-09-10T18:35:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: from='client.34121 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]': finished 2024-09-10T18:35:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/389270681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3621928698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]: dispatch 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: from='client.34121 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]: dispatch 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: from='client.34121 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]': finished 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/389270681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3621928698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]: dispatch 2024-09-10T18:35:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: from='client.34121 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]: dispatch 2024-09-10T18:35:11.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: from='client.34121 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01"}]': finished 2024-09-10T18:35:11.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:35:11.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:11.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/389270681' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:13.812 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:13 smithi131 ceph-mon[29369]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:13.812 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:13 smithi131 ceph-mon[40885]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:13 smithi175 ceph-mon[33399]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:15.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:15 smithi131 ceph-mon[29369]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:15.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:15 smithi131 ceph-mon[40885]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:15 smithi175 ceph-mon[33399]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[29369]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:35:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[29369]: Deploying daemon osd.0 on smithi131 2024-09-10T18:35:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[40885]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:17.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:35:17.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:17.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:17 smithi131 ceph-mon[40885]: Deploying daemon osd.0 on smithi131 2024-09-10T18:35:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:17 smithi175 ceph-mon[33399]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:17 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:35:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:17 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:17 smithi175 ceph-mon[33399]: Deploying daemon osd.0 on smithi131 2024-09-10T18:35:19.720 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:19 smithi131 ceph-mon[29369]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:19.720 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:19 smithi131 ceph-mon[40885]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:19 smithi175 ceph-mon[33399]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:21.691 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:21 smithi131 ceph-mon[29369]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:21.692 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:21 smithi131 ceph-mon[40885]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:21 smithi175 ceph-mon[33399]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:23.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:23 smithi131 ceph-mon[40885]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:23.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:23 smithi131 ceph-mon[29369]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:24.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:23 smithi175 ceph-mon[33399]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:25.281 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 0 on host 'smithi131' 2024-09-10T18:35:25.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.279+0000 7fb633fff640 1 -- 172.21.15.131:0/3063165190 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3823342490) 0x7fb638000bc0 con 0x7fb63403a800 2024-09-10T18:35:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.281+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 >> v1:172.21.15.131:6800/3123609852 conn(0x7fb63403a800 legacy=0x7fb63403ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.281+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 >> v1:172.21.15.175:6789/0 conn(0x7fb64c100bc0 legacy=0x7fb64c112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.281+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 shutdown_connections 2024-09-10T18:35:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.281+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 >> 172.21.15.131:0/3063165190 conn(0x7fb64c0f63c0 msgr2=0x7fb64c0f6790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.281+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 shutdown_connections 2024-09-10T18:35:25.283 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:25.281+0000 7fb6520da640 1 -- 172.21.15.131:0/3063165190 wait complete. 2024-09-10T18:35:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[40885]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:25.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:25 smithi131 ceph-mon[29369]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:25 smithi175 ceph-mon[33399]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:35:25.962 DEBUG:teuthology.orchestra.run.smithi131:osd.0> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:35:25.964 INFO:tasks.cephadm:Deploying osd.1 on smithi131 with /dev/vg_nvme/lv_3... 2024-09-10T18:35:25.965 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:35:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:35:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:35:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:26.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:26.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:26.579 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0[45274]: 2024-09-10T18:35:26.260+0000 7f16b597c640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:35:26.579 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:35:26 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0[45274]: 2024-09-10T18:35:26.268+0000 7f16ae7af640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:26.715 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:27.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[29369]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:35:27.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[29369]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:35:27.525 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[29369]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[40885]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[40885]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:27.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:27 smithi131 ceph-mon[40885]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:27 smithi175 ceph-mon[33399]: from='osd.0 v1:172.21.15.131:6801/437930310' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:35:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:27 smithi175 ceph-mon[33399]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:35:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:27 smithi175 ceph-mon[33399]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[29369]: osd.0 v1:172.21.15.131:6801/437930310 boot 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[29369]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:35:28.564 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:28.565 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[40885]: osd.0 v1:172.21.15.131:6801/437930310 boot 2024-09-10T18:35:28.565 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[40885]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:35:28.565 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:28 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:28 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:35:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:28 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:35:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:28 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:28 smithi175 ceph-mon[33399]: osd.0 v1:172.21.15.131:6801/437930310 boot 2024-09-10T18:35:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:28 smithi175 ceph-mon[33399]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:35:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:28 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:35:29.525 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:35:29.571 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:29 smithi131 ceph-mon[29369]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:35:29.571 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:29 smithi131 ceph-mon[29369]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:29.572 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:29 smithi131 ceph-mon[40885]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:35:29.572 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:29 smithi131 ceph-mon[40885]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:29.618 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:35:29.631 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-10T18:35:29.631 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-10T18:35:29.631 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112787 s, 930 MB/s 2024-09-10T18:35:29.631 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-10T18:35:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:29 smithi175 ceph-mon[33399]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:35:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:29 smithi175 ceph-mon[33399]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:30.451 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_3 2024-09-10T18:35:30.691 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi131 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi131 to 16064M 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi131 2024-09-10T18:35:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi131 to 16064M 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:30 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi131 2024-09-10T18:35:31.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:31.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi131 to 16064M 2024-09-10T18:35:31.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:31.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:31.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:31.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:30 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:32.100 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:31 smithi131 ceph-mon[29369]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:32.100 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:31 smithi131 ceph-mon[40885]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:32.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:31 smithi175 ceph-mon[33399]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:32.367 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.362+0000 7f9278b27640 1 -- 172.21.15.131:0/1592414047 >> v1:172.21.15.131:6789/0 conn(0x7f92740fb4a0 legacy=0x7f92740fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.362+0000 7f9278b27640 1 -- 172.21.15.131:0/1592414047 shutdown_connections 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.362+0000 7f9278b27640 1 -- 172.21.15.131:0/1592414047 >> 172.21.15.131:0/1592414047 conn(0x7f92740f6c60 msgr2=0x7f92740f9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.362+0000 7f9278b27640 1 -- 172.21.15.131:0/1592414047 shutdown_connections 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.362+0000 7f9278b27640 1 -- 172.21.15.131:0/1592414047 wait complete. 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9278b27640 1 Processor -- start 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9278b27640 1 -- start start 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9278b27640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f927410d0f0 con 0x7f92741035c0 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9278b27640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f92741188f0 con 0x7f92740ff9a0 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9278b27640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9274119a90 con 0x7f92740fb4a0 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f927259c640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f92740fb4a0 0x7f927410c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:57562/0 (socket says 172.21.15.131:57562) 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f927259c640 1 -- 172.21.15.131:0/1111751968 learned_addr learned my addr 172.21.15.131:0/1111751968 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020813556 0 0) 0x7f9274119a90 con 0x7f92740fb4a0 2024-09-10T18:35:32.368 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f925c003200 con 0x7f92740fb4a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2177181811 0 0) 0x7f927410d0f0 con 0x7f92741035c0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9274119a90 con 0x7f92741035c0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658467097 0 0) 0x7f92741188f0 con 0x7f92740ff9a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f927410d0f0 con 0x7f92740ff9a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1664408902 0 0) 0x7f925c003200 con 0x7f92740fb4a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92741188f0 con 0x7f92740fb4a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9264001430 con 0x7f92740fb4a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 874277348 0 0) 0x7f9274119a90 con 0x7f92741035c0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f925c003200 con 0x7f92741035c0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.363+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f926c001b40 con 0x7f92741035c0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2992813438 0 0) 0x7f92741188f0 con 0x7f92740fb4a0 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 >> v1:172.21.15.175:6789/0 conn(0x7f92740ff9a0 legacy=0x7f9274112a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.369 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 >> v1:172.21.15.131:6789/0 conn(0x7f92741035c0 legacy=0x7f92741161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927411ac70 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9274117940 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9274117e10 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f92640025a0 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.364+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f92640036d0 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.365+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9254005040 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.365+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 864645932 0 0) 0x7f92640038b0 con 0x7f92740fb4a0 2024-09-10T18:35:32.370 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.366+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1537+0+0 (unknown 3590186801 0 0) 0x7f9264088ed0 con 0x7f92740fb4a0 2024-09-10T18:35:32.371 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.370+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f9264056f10 con 0x7f92740fb4a0 2024-09-10T18:35:32.535 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:32.533+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9254000bc0 con 0x7f925c06fd20 2024-09-10T18:35:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:32 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:32 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:32 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:33.187 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:32 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:33.187 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:32 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:33.187 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:32 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:33.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:33.187 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:33.188 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:33.916 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:33 smithi131 ceph-mon[29369]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:33.916 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:33 smithi131 ceph-mon[29369]: from='client.34142 v1:172.21.15.131:0/1111751968' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:33.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:33 smithi131 ceph-mon[40885]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:33.916 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:33 smithi131 ceph-mon[40885]: from='client.34142 v1:172.21.15.131:0/1111751968' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:34.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:33 smithi175 ceph-mon[33399]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:34.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:33 smithi175 ceph-mon[33399]: from='client.34142 v1:172.21.15.131:0/1111751968' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:35.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1010950356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]: dispatch 2024-09-10T18:35:35.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:34 smithi175 ceph-mon[33399]: from='client.34148 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]: dispatch 2024-09-10T18:35:35.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:34 smithi175 ceph-mon[33399]: from='client.34148 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]': finished 2024-09-10T18:35:35.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:34 smithi175 ceph-mon[33399]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:35:35.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:34 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:35.191 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1010950356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]: dispatch 2024-09-10T18:35:35.191 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[40885]: from='client.34148 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]: dispatch 2024-09-10T18:35:35.191 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[40885]: from='client.34148 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]': finished 2024-09-10T18:35:35.191 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[40885]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:35:35.192 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:35.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1010950356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]: dispatch 2024-09-10T18:35:35.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[29369]: from='client.34148 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]: dispatch 2024-09-10T18:35:35.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[29369]: from='client.34148 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "20feda32-c9c6-42f4-8226-1652eccf014e"}]': finished 2024-09-10T18:35:35.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[29369]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:35:35.192 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:34 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:35 smithi175 ceph-mon[33399]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/485942083' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:36.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:35 smithi131 ceph-mon[29369]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:36.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/485942083' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:36.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:35 smithi131 ceph-mon[40885]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:36.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/485942083' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:37 smithi175 ceph-mon[33399]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:38.274 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:37 smithi131 ceph-mon[29369]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:38.274 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:37 smithi131 ceph-mon[40885]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:39.969 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:39 smithi131 ceph-mon[29369]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:39.969 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:39 smithi131 ceph-mon[40885]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:39 smithi175 ceph-mon[33399]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:41 smithi131 ceph-mon[40885]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:35:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:41 smithi131 ceph-mon[29369]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:35:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:42.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:41 smithi175 ceph-mon[33399]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:42.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:35:42.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:42.968 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:42 smithi131 ceph-mon[29369]: Deploying daemon osd.1 on smithi131 2024-09-10T18:35:42.968 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:42 smithi131 ceph-mon[40885]: Deploying daemon osd.1 on smithi131 2024-09-10T18:35:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:42 smithi175 ceph-mon[33399]: Deploying daemon osd.1 on smithi131 2024-09-10T18:35:44.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:43 smithi175 ceph-mon[33399]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:44.236 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:43 smithi131 ceph-mon[29369]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:44.236 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:43 smithi131 ceph-mon[40885]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:45 smithi175 ceph-mon[33399]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:45 smithi131 ceph-mon[29369]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:46.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:45 smithi131 ceph-mon[40885]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:47.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:47.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:47.240 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:47.240 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:47.240 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:47 smithi131 ceph-mon[29369]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:47 smithi131 ceph-mon[40885]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:47 smithi175 ceph-mon[33399]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:49.263 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.261+0000 7f9262ffd640 1 -- 172.21.15.131:0/1111751968 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3220602158) 0x7f9254000bc0 con 0x7f925c06fd20 2024-09-10T18:35:49.264 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 1 on host 'smithi131' 2024-09-10T18:35:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.264+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 >> v1:172.21.15.131:6800/3123609852 conn(0x7f925c06fd20 legacy=0x7f925c0721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.264+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 >> v1:172.21.15.131:6790/0 conn(0x7f92740fb4a0 legacy=0x7f927410c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.264+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 shutdown_connections 2024-09-10T18:35:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.264+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 >> 172.21.15.131:0/1111751968 conn(0x7f92740f6c60 msgr2=0x7f92740663f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.264+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 shutdown_connections 2024-09-10T18:35:49.266 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:49.264+0000 7f9278b27640 1 -- 172.21.15.131:0/1111751968 wait complete. 2024-09-10T18:35:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='osd.1 v1:172.21.15.131:6805/2444922616' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='osd.1 v1:172.21.15.131:6805/2444922616' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.867 DEBUG:teuthology.orchestra.run.smithi131:osd.1> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:35:49.869 INFO:tasks.cephadm:Deploying osd.2 on smithi131 with /dev/vg_nvme/lv_2... 2024-09-10T18:35:49.869 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:35:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:49.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='osd.1 v1:172.21.15.131:6805/2444922616' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:35:49.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:35:49.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:49.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:49.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:50.694 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[29369]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:35:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[29369]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:35:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[29369]: from='osd.1 v1:172.21.15.131:6805/2444922616' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[29369]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[29369]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[40885]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[40885]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[40885]: from='osd.1 v1:172.21.15.131:6805/2444922616' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[40885]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:51.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-mon[40885]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:51.079 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1[51510]: 2024-09-10T18:35:50.985+0000 7fd37bc37640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T18:35:51.079 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:35:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1[51510]: 2024-09-10T18:35:50.987+0000 7fd37726f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:35:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:50 smithi175 ceph-mon[33399]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:35:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:50 smithi175 ceph-mon[33399]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:35:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:50 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:50 smithi175 ceph-mon[33399]: from='osd.1 v1:172.21.15.131:6805/2444922616' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:50 smithi175 ceph-mon[33399]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:35:51.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:50 smithi175 ceph-mon[33399]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[29369]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:35:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[29369]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:35:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[40885]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:35:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[40885]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:35:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:51 smithi175 ceph-mon[33399]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:35:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:51 smithi175 ceph-mon[33399]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:35:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:52.930 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:35:53.020 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:35:53.034 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-10T18:35:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-10T18:35:53.034 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012534 s, 837 MB/s 2024-09-10T18:35:53.034 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-10T18:35:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:35:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:35:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[29369]: osd.1 v1:172.21.15.131:6805/2444922616 boot 2024-09-10T18:35:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[29369]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:35:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[29369]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:35:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:35:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[40885]: osd.1 v1:172.21.15.131:6805/2444922616 boot 2024-09-10T18:35:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[40885]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:35:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:52 smithi131 ceph-mon[40885]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:53.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:52 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:35:53.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:52 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:35:53.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:52 smithi175 ceph-mon[33399]: osd.1 v1:172.21.15.131:6805/2444922616 boot 2024-09-10T18:35:53.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:52 smithi175 ceph-mon[33399]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:35:53.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:52 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:35:53.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:52 smithi175 ceph-mon[33399]: pgmap v58: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:35:53.781 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_2 2024-09-10T18:35:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:54 smithi131 ceph-mon[29369]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:35:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:54 smithi131 ceph-mon[40885]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:35:54.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:54 smithi175 ceph-mon[33399]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:35:54.655 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi131 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi131 to 8032M 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi131 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi131 to 8032M 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:55.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi131 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi131 to 8032M 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:55.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:35:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.389+0000 7f006dca8640 1 -- 172.21.15.131:0/1049298053 >> v1:172.21.15.131:6790/0 conn(0x7f0068103570 legacy=0x7f0068105a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.389+0000 7f006dca8640 1 -- 172.21.15.131:0/1049298053 shutdown_connections 2024-09-10T18:35:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.389+0000 7f006dca8640 1 -- 172.21.15.131:0/1049298053 >> 172.21.15.131:0/1049298053 conn(0x7f00680f6c10 msgr2=0x7f00680f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:35:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.389+0000 7f006dca8640 1 -- 172.21.15.131:0/1049298053 shutdown_connections 2024-09-10T18:35:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.389+0000 7f006dca8640 1 -- 172.21.15.131:0/1049298053 wait complete. 2024-09-10T18:35:56.402 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.389+0000 7f006dca8640 1 Processor -- start 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f006dca8640 1 -- start start 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f006dca8640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f006810bfc0 con 0x7f00680ff950 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f006dca8640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f00681177c0 con 0x7f0068103570 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f006dca8640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0068118960 con 0x7f00680fb450 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f00677fe640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f00680fb450 0x7f006810b430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:54040/0 (socket says 172.21.15.131:54040) 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f00677fe640 1 -- 172.21.15.131:0/271546731 learned_addr learned my addr 172.21.15.131:0/271546731 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f0066ffd640 1 --1- 172.21.15.131:0/271546731 >> v1:172.21.15.131:6789/0 conn(0x7f00680ff950 0x7f0068111960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49354/0 (socket says 172.21.15.131:49354) 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1346595854 0 0) 0x7f0068118960 con 0x7f00680fb450 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00500034e0 con 0x7f00680fb450 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 726138789 0 0) 0x7f006810bfc0 con 0x7f00680ff950 2024-09-10T18:35:56.403 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0068118960 con 0x7f00680ff950 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824948714 0 0) 0x7f00681177c0 con 0x7f0068103570 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f006810bfc0 con 0x7f0068103570 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3113884647 0 0) 0x7f00500034e0 con 0x7f00680fb450 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00681177c0 con 0x7f00680fb450 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.390+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f00580019d0 con 0x7f00680fb450 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 955426557 0 0) 0x7f0068118960 con 0x7f00680ff950 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00500034e0 con 0x7f00680ff950 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f005c0018c0 con 0x7f00680ff950 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2391200431 0 0) 0x7f006810bfc0 con 0x7f0068103570 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0068118960 con 0x7f0068103570 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f00600017b0 con 0x7f0068103570 2024-09-10T18:35:56.404 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3257030073 0 0) 0x7f00681177c0 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 >> v1:172.21.15.175:6789/0 conn(0x7f0068103570 legacy=0x7f0068115090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 >> v1:172.21.15.131:6789/0 conn(0x7f00680ff950 legacy=0x7f0068111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0068119b40 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0068118b50 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0068119020 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0058002450 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.391+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0058005a60 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.392+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0054005040 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.392+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 14) v1 ==== 90690+0+0 (unknown 864645932 0 0) 0x7f0058002000 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.393+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1829+0+0 (unknown 336610190 0 0) 0x7f0058088bc0 con 0x7f00680fb450 2024-09-10T18:35:56.405 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.397+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0058056a80 con 0x7f00680fb450 2024-09-10T18:35:56.562 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:35:56.560+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0054000bc0 con 0x7f005006fc30 2024-09-10T18:35:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[29369]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[29369]: from='client.34169 v1:172.21.15.131:0/271546731' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[40885]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[40885]: from='client.34169 v1:172.21.15.131:0/271546731' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:57 smithi175 ceph-mon[33399]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:57 smithi175 ceph-mon[33399]: from='client.34169 v1:172.21.15.131:0/271546731' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:35:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:35:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:59.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:59.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2178158320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]: dispatch 2024-09-10T18:35:59.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: from='client.34175 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]: dispatch 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]': finished 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1691688207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2178158320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]: dispatch 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: from='client.34175 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]: dispatch 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]': finished 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:35:59.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:35:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1691688207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:35:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:35:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2178158320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]: dispatch 2024-09-10T18:35:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: from='client.34175 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]: dispatch 2024-09-10T18:35:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: from='client.34175 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd6bffb1-379f-41bc-9600-f7aed3a9c7d8"}]': finished 2024-09-10T18:35:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:35:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:35:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:35:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1691688207' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:01 smithi131 ceph-mon[29369]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:01 smithi131 ceph-mon[40885]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:01 smithi175 ceph-mon[33399]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:03.663 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:03 smithi131 ceph-mon[29369]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:03.663 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:03 smithi131 ceph-mon[40885]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:03 smithi175 ceph-mon[33399]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:05.663 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[29369]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[29369]: Deploying daemon osd.2 on smithi131 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[40885]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:05.664 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:05 smithi131 ceph-mon[40885]: Deploying daemon osd.2 on smithi131 2024-09-10T18:36:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:05 smithi175 ceph-mon[33399]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:36:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:05 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:05 smithi175 ceph-mon[33399]: Deploying daemon osd.2 on smithi131 2024-09-10T18:36:07.760 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:07 smithi131 ceph-mon[29369]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:07.760 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:07 smithi131 ceph-mon[40885]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:07 smithi175 ceph-mon[33399]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:09.635 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:09 smithi131 ceph-mon[29369]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:09.636 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:09 smithi131 ceph-mon[40885]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:09 smithi175 ceph-mon[33399]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:10 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:10 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:10 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:10.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:10 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:10 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:10 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:10 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:10 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:10 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:11 smithi175 ceph-mon[33399]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:12.212 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:11 smithi131 ceph-mon[29369]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:12.213 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:11 smithi131 ceph-mon[40885]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:12.993 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 2 on host 'smithi131' 2024-09-10T18:36:12.994 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.991+0000 7f004ffff640 1 -- 172.21.15.131:0/271546731 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1539854066) 0x7f0054000bc0 con 0x7f005006fc30 2024-09-10T18:36:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.993+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 >> v1:172.21.15.131:6800/3123609852 conn(0x7f005006fc30 legacy=0x7f00500720e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:12.995 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.993+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 >> v1:172.21.15.131:6790/0 conn(0x7f00680fb450 legacy=0x7f006810b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.993+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 shutdown_connections 2024-09-10T18:36:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.994+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 >> 172.21.15.131:0/271546731 conn(0x7f00680f6c10 msgr2=0x7f00681059c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.994+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 shutdown_connections 2024-09-10T18:36:12.996 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:12.994+0000 7f006dca8640 1 -- 172.21.15.131:0/271546731 wait complete. 2024-09-10T18:36:13.610 DEBUG:teuthology.orchestra.run.smithi131:osd.2> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:36:13.612 INFO:tasks.cephadm:Deploying osd.3 on smithi131 with /dev/vg_nvme/lv_1... 2024-09-10T18:36:13.613 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:36:13.910 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:14.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[29369]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:14.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[29369]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:36:14.169 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[40885]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[40885]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.170 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:13 smithi175 ceph-mon[33399]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:13 smithi175 ceph-mon[33399]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:36:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:15.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[40885]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:36:15.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[40885]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:36:15.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:15.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[40885]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:36:15.005 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[40885]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:15.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[29369]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:36:15.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[29369]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:36:15.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:15.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[29369]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:36:15.006 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:14 smithi131 ceph-mon[29369]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:15.306 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:15 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:36:15.002+0000 7f6e8e8e6640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:36:15.306 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:15 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:36:15.005+0000 7f6e8af20640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:36:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:15 smithi175 ceph-mon[33399]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:36:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:15 smithi175 ceph-mon[33399]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:36:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:15 smithi175 ceph-mon[33399]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:36:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:15 smithi175 ceph-mon[33399]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:36:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:36:16.059 INFO:teuthology.orchestra.run.smithi131.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:36:16.074 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10+0 records in 2024-09-10T18:36:16.074 INFO:teuthology.orchestra.run.smithi131.stderr:10+0 records out 2024-09-10T18:36:16.074 INFO:teuthology.orchestra.run.smithi131.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127832 s, 820 MB/s 2024-09-10T18:36:16.075 INFO:teuthology.orchestra.run.smithi131.stderr:--> Zapping successful for: 2024-09-10T18:36:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[29369]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:36:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[29369]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:36:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[40885]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:36:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[40885]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:36:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:16 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:16 smithi175 ceph-mon[33399]: from='osd.2 v1:172.21.15.131:6809/1436967876' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:36:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:16 smithi175 ceph-mon[33399]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:36:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:16 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:16 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:16.793 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi131:vg_nvme/lv_1 2024-09-10T18:36:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:36:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:36:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: osd.2 v1:172.21.15.131:6809/1436967876 boot 2024-09-10T18:36:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:36:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: osd.2 v1:172.21.15.131:6809/1436967876 boot 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:17.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:17 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: osd.2 v1:172.21.15.131:6809/1436967876 boot 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: pgmap v75: 0 pgs: ; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:17 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:36:17.765 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:18.177 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:36:18.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:18 smithi131 ceph-mon[29369]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:36:18.178 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:18 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:36:18.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:18 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:36:18.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:18 smithi131 ceph-mon[40885]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:36:18.178 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:18 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:36:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:18 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:36:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:18 smithi175 ceph-mon[33399]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:36:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:18 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:36:18.765 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61825]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:36:18.765 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61825]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:18.765 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61825]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:18.765 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61825]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:19.029 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61834]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:36:19.029 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61834]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:19.029 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61834]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:19.029 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:36:18 smithi131 sudo[61834]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:36:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[29369]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:36:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[29369]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:19.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:36:19.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[40885]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:36:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[40885]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:19 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:19.328 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61846]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:36:19.328 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61846]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:19.328 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61846]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:19.328 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61846]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:19 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:36:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:19 smithi175 ceph-mon[33399]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:36:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:19 smithi175 ceph-mon[33399]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:19 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:19.650 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61912]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:36:19.650 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61912]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:19.650 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61912]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:19.650 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61912]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:19.913 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61935]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:36:19.913 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61935]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:19.913 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61935]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:19.913 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61935]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:20.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 -- 172.21.15.131:0/108937857 >> v1:172.21.15.131:6790/0 conn(0x7f653c104690 legacy=0x7f653c106b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:20.005 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 -- 172.21.15.131:0/108937857 shutdown_connections 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 -- 172.21.15.131:0/108937857 >> 172.21.15.131:0/108937857 conn(0x7f653c0f6c20 msgr2=0x7f653c0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 -- 172.21.15.131:0/108937857 shutdown_connections 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 -- 172.21.15.131:0/108937857 wait complete. 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 Processor -- start 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.001+0000 7f654387d640 1 -- start start 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f654387d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f653c10d280 con 0x7f653c104690 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f654387d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f653c118a80 con 0x7f653c1008b0 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f654387d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f653c119c20 con 0x7f653c065b30 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6541619640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f653c065b30 0x7f653c10c6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:49580/0 (socket says 172.21.15.131:49580) 2024-09-10T18:36:20.006 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6541619640 1 -- 172.21.15.131:0/3493416789 learned_addr learned my addr 172.21.15.131:0/3493416789 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525241824 0 0) 0x7f653c119c20 con 0x7f653c065b30 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6528003720 con 0x7f653c065b30 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187293639 0 0) 0x7f653c118a80 con 0x7f653c1008b0 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f653c119c20 con 0x7f653c1008b0 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2770975022 0 0) 0x7f6528003720 con 0x7f653c065b30 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f653c118a80 con 0x7f653c065b30 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2658900869 0 0) 0x7f653c119c20 con 0x7f653c1008b0 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6528003720 con 0x7f653c1008b0 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f65380029b0 con 0x7f653c065b30 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f652c001770 con 0x7f653c1008b0 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3772055759 0 0) 0x7f653c118a80 con 0x7f653c065b30 2024-09-10T18:36:20.007 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 >> v1:172.21.15.175:6789/0 conn(0x7f653c1008b0 legacy=0x7f653c112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 >> v1:172.21.15.131:6789/0 conn(0x7f653c104690 legacy=0x7f653c116350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.002+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f653c11ae00 con 0x7f653c065b30 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.003+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653c117ad0 con 0x7f653c065b30 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.003+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f653c117fa0 con 0x7f653c065b30 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.003+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f6538002ba0 con 0x7f653c065b30 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.003+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f6538005a80 con 0x7f653c065b30 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.004+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6520005040 con 0x7f653c065b30 2024-09-10T18:36:20.008 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.004+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f6538005c00 con 0x7f653c065b30 2024-09-10T18:36:20.009 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.005+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 2507+0+0 (unknown 4095173491 0 0) 0x7f6538088ea0 con 0x7f653c065b30 2024-09-10T18:36:20.010 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.009+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f65380565b0 con 0x7f653c065b30 2024-09-10T18:36:20.169 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61957]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:36:20.169 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61957]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:20.169 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61957]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:20.169 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:36:19 smithi131 sudo[61957]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:20.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:20.177+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f6520000bc0 con 0x7f6528078450 2024-09-10T18:36:20.423 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61986]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:36:20.423 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61986]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:20.423 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61986]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:20.424 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61986]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:20.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:36:20.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:36:20.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.688 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:20.689 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36020]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:36:20.690 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36020]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:20.690 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36020]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:20.690 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36020]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:20.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:36:20.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:36:20.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:20.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61990]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61990]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61990]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 sudo[61990]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:20.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:20 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:21.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36023]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:36:21.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36023]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:21.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36023]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:21.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:20 smithi175 sudo[36023]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:21.458 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62151]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:36:21.458 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62151]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:21.458 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62151]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:21.458 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62151]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:21.803 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi131 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi131 to 5354M 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='client.34202 v1:172.21.15.131:0/3493416789' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:21.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62154]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62154]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62154]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 sudo[62154]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi131 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi131 to 5354M 2024-09-10T18:36:21.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='client.34202 v1:172.21.15.131:0/3493416789' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:21.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:21.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi131 2024-09-10T18:36:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi131 to 5354M 2024-09-10T18:36:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='client.34202 v1:172.21.15.131:0/3493416789' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi131:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:21.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:22 smithi131 ceph-mon[29369]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:22 smithi131 ceph-mon[29369]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:22 smithi131 ceph-mon[40885]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:22 smithi131 ceph-mon[40885]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:22 smithi175 ceph-mon[33399]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:36:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:22 smithi175 ceph-mon[33399]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[29369]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3152162958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b888d2e-d94c-4cd8-afad-498c49a22a4b"}]: dispatch 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3152162958' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b888d2e-d94c-4cd8-afad-498c49a22a4b"}]': finished 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[29369]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3438336965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[40885]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3152162958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b888d2e-d94c-4cd8-afad-498c49a22a4b"}]: dispatch 2024-09-10T18:36:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3152162958' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b888d2e-d94c-4cd8-afad-498c49a22a4b"}]': finished 2024-09-10T18:36:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[40885]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:36:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3438336965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:23 smithi175 ceph-mon[33399]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3152162958' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5b888d2e-d94c-4cd8-afad-498c49a22a4b"}]: dispatch 2024-09-10T18:36:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3152162958' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5b888d2e-d94c-4cd8-afad-498c49a22a4b"}]': finished 2024-09-10T18:36:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:23 smithi175 ceph-mon[33399]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:36:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:23 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3438336965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:25.769 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:25 smithi131 ceph-mon[29369]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:25.770 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:25 smithi131 ceph-mon[40885]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:25 smithi175 ceph-mon[33399]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:27.803 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:27 smithi131 ceph-mon[29369]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:27.803 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:27 smithi131 ceph-mon[40885]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:27 smithi175 ceph-mon[33399]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:29.544 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[29369]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:29.544 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:36:29.544 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[29369]: Deploying daemon osd.3 on smithi131 2024-09-10T18:36:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[40885]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:36:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.545 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:29 smithi131 ceph-mon[40885]: Deploying daemon osd.3 on smithi131 2024-09-10T18:36:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:29 smithi175 ceph-mon[33399]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:29 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:36:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:29 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:29 smithi175 ceph-mon[33399]: Deploying daemon osd.3 on smithi131 2024-09-10T18:36:31.522 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:31 smithi131 ceph-mon[29369]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:31.523 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:31 smithi131 ceph-mon[40885]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:31 smithi175 ceph-mon[33399]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:33 smithi131 ceph-mon[29369]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:33.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:33 smithi131 ceph-mon[40885]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:33 smithi175 ceph-mon[33399]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:34 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:34 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:34 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:34 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:34 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:34 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:34 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:34 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:34 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:35 smithi131 ceph-mon[29369]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:35 smithi131 ceph-mon[40885]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:35 smithi175 ceph-mon[33399]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:37.279 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.277+0000 7f6531ffb640 1 -- 172.21.15.131:0/3493416789 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 131964998) 0x7f6520000bc0 con 0x7f6528078450 2024-09-10T18:36:37.280 INFO:teuthology.orchestra.run.smithi131.stdout:Created osd(s) 3 on host 'smithi131' 2024-09-10T18:36:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.279+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 >> v1:172.21.15.131:6800/3123609852 conn(0x7f6528078450 legacy=0x7f652807a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:37.281 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.279+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 >> v1:172.21.15.131:6790/0 conn(0x7f653c065b30 legacy=0x7f653c10c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.279+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 shutdown_connections 2024-09-10T18:36:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.279+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 >> 172.21.15.131:0/3493416789 conn(0x7f653c0f6c20 msgr2=0x7f653c104af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.280+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 shutdown_connections 2024-09-10T18:36:37.282 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:36:37.280+0000 7f654387d640 1 -- 172.21.15.131:0/3493416789 wait complete. 2024-09-10T18:36:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:37.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:36:37.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:37.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:37.985 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:37.986 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:37.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:38.014 DEBUG:teuthology.orchestra.run.smithi131:osd.3> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:36:38.017 INFO:tasks.cephadm:Deploying osd.4 on smithi175 with /dev/vg_nvme/lv_4... 2024-09-10T18:36:38.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:36:38.264 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:36:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:38 smithi175 ceph-mon[33399]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:36:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:38 smithi175 ceph-mon[33399]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:36:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:38 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:38 smithi175 ceph-mon[33399]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:36:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[29369]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:36:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[29369]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:36:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[29369]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:36:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[40885]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:36:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[40885]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:36:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-mon[40885]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]: dispatch 2024-09-10T18:36:39.078 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:36:38.740+0000 7fbb3849c640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:36:39.078 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:36:38 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:36:38.747+0000 7fbb33ad4640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:36:40.071 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:36:40.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:39 smithi175 ceph-mon[33399]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:40.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:39 smithi175 ceph-mon[33399]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:36:40.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:39 smithi175 ceph-mon[33399]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:36:40.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:40.071 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:39 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[29369]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[29369]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:36:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[29369]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:36:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[40885]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[40885]: from='osd.3 v1:172.21.15.131:6813/2850347725' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi131", "root=default"]}]': finished 2024-09-10T18:36:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[40885]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:36:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:39 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:40.145 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:36:40.159 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-10T18:36:40.159 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-10T18:36:40.159 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115406 s, 909 MB/s 2024-09-10T18:36:40.159 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-10T18:36:40.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_4 2024-09-10T18:36:40.959 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:36:41.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:40 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:36:41.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:40 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:36:41.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:40 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:41.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:40 smithi175 ceph-mon[33399]: osd.3 v1:172.21.15.131:6813/2850347725 boot 2024-09-10T18:36:41.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:40 smithi175 ceph-mon[33399]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:36:41.070 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:40 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:36:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:36:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[40885]: osd.3 v1:172.21.15.131:6813/2850347725 boot 2024-09-10T18:36:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[40885]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:36:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:36:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:36:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[29369]: osd.3 v1:172.21.15.131:6813/2850347725 boot 2024-09-10T18:36:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[29369]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:36:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:40 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi131 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.025 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi131 to 4016M 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:42.026 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi131 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi131 to 4016M 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi131 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi131 to 4016M 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:42.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.532+0000 7fa6ead98640 1 -- 172.21.15.175:0/444795475 >> v1:172.21.15.175:6789/0 conn(0x7fa6e40ff950 legacy=0x7fa6e4101d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:42.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.532+0000 7fa6ead98640 1 -- 172.21.15.175:0/444795475 shutdown_connections 2024-09-10T18:36:42.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.532+0000 7fa6ead98640 1 -- 172.21.15.175:0/444795475 >> 172.21.15.175:0/444795475 conn(0x7fa6e40f6c10 msgr2=0x7fa6e40f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:42.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.532+0000 7fa6ead98640 1 -- 172.21.15.175:0/444795475 shutdown_connections 2024-09-10T18:36:42.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.532+0000 7fa6ead98640 1 -- 172.21.15.175:0/444795475 wait complete. 2024-09-10T18:36:42.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6ead98640 1 Processor -- start 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6ead98640 1 -- start start 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6ead98640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6e4117500 con 0x7fa6e4103570 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6ead98640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6e4118700 con 0x7fa6e40ff950 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6ead98640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa6e4119900 con 0x7fa6e40fb450 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e3fff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fa6e40ff950 0x7fa6e41122d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:49516/0 (socket says 172.21.15.175:49516) 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e3fff640 1 -- 172.21.15.175:0/3478479709 learned_addr learned my addr 172.21.15.175:0/3478479709 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e8b34640 1 --1- 172.21.15.175:0/3478479709 >> v1:172.21.15.131:6790/0 conn(0x7fa6e40fb450 0x7fa6e4066060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.175:52996/0 (socket says 172.21.15.175:52996) 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128792738 0 0) 0x7fa6e4118700 con 0x7fa6e40ff950 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6d4003720 con 0x7fa6e40ff950 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903579676 0 0) 0x7fa6e4117500 con 0x7fa6e4103570 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4118700 con 0x7fa6e4103570 2024-09-10T18:36:42.562 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701728552 0 0) 0x7fa6e4119900 con 0x7fa6e40fb450 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.533+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4117500 con 0x7fa6e40fb450 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3116856186 0 0) 0x7fa6e4118700 con 0x7fa6e4103570 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6e4119900 con 0x7fa6e4103570 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3818506464 0 0) 0x7fa6d4003720 con 0x7fa6e40ff950 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6e4118700 con 0x7fa6e40ff950 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa6dc001bc0 con 0x7fa6e4103570 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa6d0002060 con 0x7fa6e40ff950 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3889485744 0 0) 0x7fa6e4117500 con 0x7fa6e40fb450 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa6d4003720 con 0x7fa6e40fb450 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa6d8001840 con 0x7fa6e40fb450 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 219900465 0 0) 0x7fa6e4119900 con 0x7fa6e4103570 2024-09-10T18:36:42.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 >> v1:172.21.15.131:6790/0 conn(0x7fa6e40fb450 legacy=0x7fa6e4066060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 >> v1:172.21.15.175:6789/0 conn(0x7fa6e40ff950 legacy=0x7fa6e41122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e411ab00 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e41176f0 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e4117bc0 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fa6dc002230 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.534+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa6dc003450 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.535+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c8005040 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.536+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7fa6dc001de0 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.536+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(26..26 src has 1..26) v4 ==== 2799+0+0 (unknown 3241870728 0 0) 0x7fa6dc086950 con 0x7fa6e4103570 2024-09-10T18:36:42.564 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.540+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fa6dc0543c0 con 0x7fa6e4103570 2024-09-10T18:36:42.703 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:42.702+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6c8000bc0 con 0x7fa6d406fee0 2024-09-10T18:36:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:42 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:42 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:42 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:42 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:43.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:42 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:43.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:42 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:43.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:42 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:43.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:42 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:43.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:42 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:44.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:43 smithi175 ceph-mon[33399]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:44.017 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:43 smithi175 ceph-mon[33399]: from='client.14328 v1:172.21.15.175:0/3478479709' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:43 smithi131 ceph-mon[29369]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:43 smithi131 ceph-mon[29369]: from='client.14328 v1:172.21.15.175:0/3478479709' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:43 smithi131 ceph-mon[40885]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:43 smithi131 ceph-mon[40885]: from='client.14328 v1:172.21.15.175:0/3478479709' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/2364439338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]: dispatch 2024-09-10T18:36:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]: dispatch 2024-09-10T18:36:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]': finished 2024-09-10T18:36:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:36:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/1991018736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/2364439338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]: dispatch 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]: dispatch 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]': finished 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:36:46.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/1991018736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/2364439338' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]: dispatch 2024-09-10T18:36:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: from='client.24238 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]: dispatch 2024-09-10T18:36:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: from='client.24238 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "638c463c-43c6-4ca5-b221-de408d307972"}]': finished 2024-09-10T18:36:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:36:46.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:36:46.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/1991018736' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:47 smithi175 ceph-mon[33399]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:47 smithi131 ceph-mon[29369]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:47 smithi131 ceph-mon[40885]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:49.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:49 smithi175 ceph-mon[33399]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:49 smithi131 ceph-mon[29369]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:49 smithi131 ceph-mon[40885]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:51.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:51 smithi175 ceph-mon[33399]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:51.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:36:51.895 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:51.896 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:51 smithi175 ceph-mon[33399]: Deploying daemon osd.4 on smithi175 2024-09-10T18:36:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[29369]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:36:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[29369]: Deploying daemon osd.4 on smithi175 2024-09-10T18:36:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[40885]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:36:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:51 smithi131 ceph-mon[40885]: Deploying daemon osd.4 on smithi175 2024-09-10T18:36:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:53 smithi131 ceph-mon[29369]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:53 smithi131 ceph-mon[40885]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:54.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:53 smithi175 ceph-mon[33399]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[29369]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[40885]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:55 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:56.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:55 smithi175 ceph-mon[33399]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:56.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:56.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:56.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:55 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.056 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:57 smithi175 ceph-mon[33399]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:58.056 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.056 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.056 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:58.056 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:58.056 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[29369]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[40885]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:36:58.876 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:58 smithi175 ceph-mon[33399]: from='osd.4 v1:172.21.15.175:6800/2653212041' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:36:58.876 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:58 smithi175 ceph-mon[33399]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:36:58.907 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.905+0000 7fa6e17fa640 1 -- 172.21.15.175:0/3478479709 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1940179538) 0x7fa6c8000bc0 con 0x7fa6d406fee0 2024-09-10T18:36:58.907 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 4 on host 'smithi175' 2024-09-10T18:36:58.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.907+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 >> v1:172.21.15.131:6800/3123609852 conn(0x7fa6d406fee0 legacy=0x7fa6d4072390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.907+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 >> v1:172.21.15.131:6789/0 conn(0x7fa6e4103570 legacy=0x7fa6e4115c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:36:58.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.908+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 shutdown_connections 2024-09-10T18:36:58.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.908+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 >> 172.21.15.175:0/3478479709 conn(0x7fa6e40f6c10 msgr2=0x7fa6e41039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:36:58.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.908+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 shutdown_connections 2024-09-10T18:36:58.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:36:58.908+0000 7fa6ead98640 1 -- 172.21.15.175:0/3478479709 wait complete. 2024-09-10T18:36:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:58 smithi131 ceph-mon[29369]: from='osd.4 v1:172.21.15.175:6800/2653212041' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:36:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:58 smithi131 ceph-mon[29369]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:36:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:58 smithi131 ceph-mon[40885]: from='osd.4 v1:172.21.15.175:6800/2653212041' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:36:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:58 smithi131 ceph-mon[40885]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:36:59.435 DEBUG:teuthology.orchestra.run.smithi175:osd.4> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:36:59.438 INFO:tasks.cephadm:Deploying osd.5 on smithi175 with /dev/vg_nvme/lv_3... 2024-09-10T18:36:59.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:37:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:37:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:37:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:37:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='osd.4 v1:172.21.15.175:6800/2653212041' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='osd.4 v1:172.21.15.175:6800/2653212041' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:00.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:00.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:36:59 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:00.177 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:36:59.800+0000 7fb397b27640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:37:00.177 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:36:59.803+0000 7fb39295e640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='osd.4 v1:172.21.15.175:6800/2653212041' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:36:59 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:00.486 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:37:00.870 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:00.870 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:37:00.870 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:00.870 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:00.870 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: osd.4 v1:172.21.15.175:6800/2653212041 boot 2024-09-10T18:37:00.871 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:37:00.871 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:00 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: osd.4 v1:172.21.15.175:6800/2653212041 boot 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:37:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: osd.4 v1:172.21.15.175:6800/2653212041 boot 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:37:01.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:00 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:37:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:01 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:37:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:01 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:37:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:01 smithi175 ceph-mon[33399]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:37:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:01 smithi175 ceph-mon[33399]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:37:02.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:37:02.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:37:02.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[29369]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:37:02.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[29369]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:37:02.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:37:02.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:37:02.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[40885]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:37:02.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:01 smithi131 ceph-mon[40885]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:37:03.225 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:37:03.304 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:37:03.319 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-10T18:37:03.319 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-10T18:37:03.319 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129682 s, 809 MB/s 2024-09-10T18:37:03.319 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-10T18:37:03.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:03.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:37:03.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi175 2024-09-10T18:37:03.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:03.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:03.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:03.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-10T18:37:03.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:03.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:03.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:03.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:03 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:03.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_3 2024-09-10T18:37:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:37:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi175 2024-09-10T18:37:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi175 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi175 to 17088M 2024-09-10T18:37:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:03 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:04.173 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 -- 172.21.15.175:0/3699216113 >> v1:172.21.15.175:6789/0 conn(0x7f38140fc570 legacy=0x7f38140fc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 -- 172.21.15.175:0/3699216113 shutdown_connections 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 -- 172.21.15.175:0/3699216113 >> 172.21.15.175:0/3699216113 conn(0x7f38140f7d70 msgr2=0x7f38140fa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 -- 172.21.15.175:0/3699216113 shutdown_connections 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 -- 172.21.15.175:0/3699216113 wait complete. 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 Processor -- start 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.739+0000 7f381953e640 1 -- start start 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f381953e640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f381410d200 con 0x7f3814100a70 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f381953e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3814118a00 con 0x7f3814104690 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f381953e640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3814119ba0 con 0x7f38140fc570 2024-09-10T18:37:05.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f38137fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f3814104690 0x7f38141162d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:47002/0 (socket says 172.21.15.175:47002) 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f38137fe640 1 -- 172.21.15.175:0/4053549056 learned_addr learned my addr 172.21.15.175:0/4053549056 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 539841870 0 0) 0x7f3814118a00 con 0x7f3814104690 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37fc003120 con 0x7f3814104690 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419572202 0 0) 0x7f3814119ba0 con 0x7f38140fc570 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3814118a00 con 0x7f38140fc570 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488248322 0 0) 0x7f381410d200 con 0x7f3814100a70 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3814119ba0 con 0x7f3814100a70 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4117367645 0 0) 0x7f37fc003120 con 0x7f3814104690 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f381410d200 con 0x7f3814104690 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f380c001c80 con 0x7f3814104690 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4240532026 0 0) 0x7f3814118a00 con 0x7f38140fc570 2024-09-10T18:37:05.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f37fc003120 con 0x7f38140fc570 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3804002c00 con 0x7f38140fc570 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2447197071 0 0) 0x7f3814119ba0 con 0x7f3814100a70 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3814118a00 con 0x7f3814100a70 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3808001730 con 0x7f3814100a70 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4271644620 0 0) 0x7f381410d200 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 >> v1:172.21.15.131:6790/0 conn(0x7f38140fc570 legacy=0x7f381410c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 >> v1:172.21.15.131:6789/0 conn(0x7f3814100a70 legacy=0x7f3814112ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381411ad80 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3814117a50 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.740+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3814117ea0 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.741+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f380c002010 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.741+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f380c005e00 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.741+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3800005040 con 0x7f3814104690 2024-09-10T18:37:05.763 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.742+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f380c002b70 con 0x7f3814104690 2024-09-10T18:37:05.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.742+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(32..32 src has 1..32) v4 ==== 3201+0+0 (unknown 1914595867 0 0) 0x7f380c089560 con 0x7f3814104690 2024-09-10T18:37:05.764 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.747+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f380c056e40 con 0x7f3814104690 2024-09-10T18:37:05.911 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:05.909+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f3800000bc0 con 0x7f37fc06fe10 2024-09-10T18:37:05.911 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:05 smithi175 ceph-mon[33399]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-10T18:37:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:05 smithi131 ceph-mon[29369]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-10T18:37:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:05 smithi131 ceph-mon[40885]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-09-10T18:37:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:06 smithi175 ceph-mon[33399]: from='client.24259 v1:172.21.15.175:0/4053549056' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:06 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:06 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:06 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[29369]: from='client.24259 v1:172.21.15.175:0/4053549056' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:07.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[40885]: from='client.24259 v1:172.21.15.175:0/4053549056' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:07.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:07.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:07.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:06 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:07 smithi175 ceph-mon[33399]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:37:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:07 smithi131 ceph-mon[29369]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:37:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:07 smithi131 ceph-mon[40885]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:37:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/2141458176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]: dispatch 2024-09-10T18:37:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[29369]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]: dispatch 2024-09-10T18:37:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[29369]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]': finished 2024-09-10T18:37:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[29369]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/3478361894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/2141458176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]: dispatch 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[40885]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]: dispatch 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[40885]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]': finished 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[40885]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/3478361894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/2141458176' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]: dispatch 2024-09-10T18:37:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:08 smithi175 ceph-mon[33399]: from='client.24265 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]: dispatch 2024-09-10T18:37:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:08 smithi175 ceph-mon[33399]: from='client.24265 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c07e1e1e-2ad2-4acb-b8d2-0a77b6661604"}]': finished 2024-09-10T18:37:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:08 smithi175 ceph-mon[33399]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:37:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:08 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/3478361894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:09 smithi131 ceph-mon[29369]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:37:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:09 smithi131 ceph-mon[40885]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:37:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:09 smithi175 ceph-mon[33399]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:37:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:11 smithi175 ceph-mon[33399]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:37:12.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:11 smithi131 ceph-mon[29369]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:37:12.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:11 smithi131 ceph-mon[40885]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:37:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:13 smithi131 ceph-mon[29369]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:37:14.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:13 smithi131 ceph-mon[40885]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:37:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:13 smithi175 ceph-mon[33399]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T18:37:15.032 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:14 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:37:15.032 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:14 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:37:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:14 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:15.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:14 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:37:15.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:14 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:15 smithi175 ceph-mon[33399]: Deploying daemon osd.5 on smithi175 2024-09-10T18:37:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:15 smithi175 ceph-mon[33399]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:15 smithi131 ceph-mon[29369]: Deploying daemon osd.5 on smithi175 2024-09-10T18:37:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:15 smithi131 ceph-mon[29369]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:15 smithi131 ceph-mon[40885]: Deploying daemon osd.5 on smithi175 2024-09-10T18:37:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:15 smithi131 ceph-mon[40885]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:18.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:17 smithi131 ceph-mon[29369]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:18.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:17 smithi131 ceph-mon[40885]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:18.120 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:17 smithi175 ceph-mon[33399]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[29369]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:20.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[40885]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:19 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:20.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:19 smithi175 ceph-mon[33399]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:20.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:19 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:20.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:19 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:20.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:19 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:21 smithi175 ceph-mon[33399]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:21.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:21 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[29369]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[40885]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:21 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.361 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.357+0000 7f37fb7fe640 1 -- 172.21.15.175:0/4053549056 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 800138470) 0x7f3800000bc0 con 0x7f37fc06fe10 2024-09-10T18:37:22.361 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 5 on host 'smithi175' 2024-09-10T18:37:22.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.359+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 >> v1:172.21.15.131:6800/3123609852 conn(0x7f37fc06fe10 legacy=0x7f37fc0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:22.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.359+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 >> v1:172.21.15.175:6789/0 conn(0x7f3814104690 legacy=0x7f38141162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:22.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.359+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 shutdown_connections 2024-09-10T18:37:22.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.359+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 >> 172.21.15.175:0/4053549056 conn(0x7f38140f7d70 msgr2=0x7f3814106b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:22.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.359+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 shutdown_connections 2024-09-10T18:37:22.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:22.359+0000 7f381953e640 1 -- 172.21.15.175:0/4053549056 wait complete. 2024-09-10T18:37:22.942 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:22 smithi175 ceph-mon[33399]: from='osd.5 v1:172.21.15.175:6804/210062365' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:37:22.942 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:22 smithi175 ceph-mon[33399]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:37:22.942 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:22.942 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.942 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:22.966 DEBUG:teuthology.orchestra.run.smithi175:osd.5> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:37:22.969 INFO:tasks.cephadm:Deploying osd.6 on smithi175 with /dev/vg_nvme/lv_2... 2024-09-10T18:37:22.969 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:37:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[29369]: from='osd.5 v1:172.21.15.175:6804/210062365' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:37:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[29369]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:37:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[40885]: from='osd.5 v1:172.21.15.175:6804/210062365' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:37:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[40885]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:37:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:23.760 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:37:24.092 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-mon[33399]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:24.092 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-mon[33399]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:37:24.092 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-mon[33399]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:37:24.092 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:24.092 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-mon[33399]: from='osd.5 v1:172.21.15.175:6804/210062365' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:24.093 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-mon[33399]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:24.093 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:37:23.838+0000 7f3860abd640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T18:37:24.093 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:37:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:37:23.842+0000 7f385b8f4640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:37:24.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[29369]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:24.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[29369]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:37:24.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[29369]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:37:24.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:24.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[29369]: from='osd.5 v1:172.21.15.175:6804/210062365' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:24.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[29369]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:24.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[40885]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:24.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[40885]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:37:24.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[40885]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:37:24.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:24.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[40885]: from='osd.5 v1:172.21.15.175:6804/210062365' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:24.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:23 smithi131 ceph-mon[40885]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: osd.5 v1:172.21.15.175:6804/210062365 boot 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:37:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: osd.5 v1:172.21.15.175:6804/210062365 boot 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:37:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: osd.5 v1:172.21.15.175:6804/210062365 boot 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:37:25.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:37:26.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:25 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:37:26.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:25 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:37:26.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:25 smithi175 ceph-mon[33399]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:26.095 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:25 smithi175 ceph-mon[33399]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[29369]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[29369]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[40885]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:37:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:25 smithi131 ceph-mon[40885]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:37:26.577 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:37:26.673 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:37:26.690 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-10T18:37:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-10T18:37:26.690 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150727 s, 696 MB/s 2024-09-10T18:37:26.690 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-10T18:37:27.567 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_2 2024-09-10T18:37:27.806 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:37:28.096 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi175 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:28.097 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:28.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:37:28.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi175 2024-09-10T18:37:28.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.130 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi175 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.131 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:28.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:28.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi175 to 8544M 2024-09-10T18:37:28.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:28.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:28.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:28.132 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.431+0000 7f3078bb2640 1 -- 172.21.15.175:0/1293012288 >> v1:172.21.15.175:6789/0 conn(0x7f30741046d0 legacy=0x7f3074106b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.431+0000 7f3078bb2640 1 -- 172.21.15.175:0/1293012288 shutdown_connections 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.431+0000 7f3078bb2640 1 -- 172.21.15.175:0/1293012288 >> 172.21.15.175:0/1293012288 conn(0x7f30740f63f0 msgr2=0x7f30740f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 -- 172.21.15.175:0/1293012288 shutdown_connections 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 -- 172.21.15.175:0/1293012288 wait complete. 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 Processor -- start 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 -- start start 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f307410d1f0 con 0x7f30740fdbd0 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f30741189f0 con 0x7f30741046d0 2024-09-10T18:37:29.444 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3078bb2640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3074119b90 con 0x7f3074100bd0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3072d9d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f30741046d0 0x7f30741162c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:45286/0 (socket says 172.21.15.175:45286) 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f3072d9d640 1 -- 172.21.15.175:0/3870903684 learned_addr learned my addr 172.21.15.175:0/3870903684 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.432+0000 7f307259c640 1 --1- 172.21.15.175:0/3870903684 >> v1:172.21.15.131:6789/0 conn(0x7f30740fdbd0 0x7f307410c660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.175:38816/0 (socket says 172.21.15.175:38816) 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927928472 0 0) 0x7f30741189f0 con 0x7f30741046d0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f305c0034e0 con 0x7f30741046d0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306479031 0 0) 0x7f3074119b90 con 0x7f3074100bd0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30741189f0 con 0x7f3074100bd0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012411064 0 0) 0x7f307410d1f0 con 0x7f30740fdbd0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3074119b90 con 0x7f30740fdbd0 2024-09-10T18:37:29.445 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1142259788 0 0) 0x7f305c0034e0 con 0x7f30741046d0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f307410d1f0 con 0x7f30741046d0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1773109117 0 0) 0x7f30741189f0 con 0x7f3074100bd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f305c0034e0 con 0x7f3074100bd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f306c002f30 con 0x7f30741046d0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3068001840 con 0x7f3074100bd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2490796820 0 0) 0x7f3074119b90 con 0x7f30740fdbd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30741189f0 con 0x7f30740fdbd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3064001870 con 0x7f30740fdbd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1885937597 0 0) 0x7f305c0034e0 con 0x7f3074100bd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 >> v1:172.21.15.175:6789/0 conn(0x7f30741046d0 legacy=0x7f30741162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 >> v1:172.21.15.131:6789/0 conn(0x7f30740fdbd0 legacy=0x7f307410c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307411ad70 con 0x7f3074100bd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3074117a40 con 0x7f3074100bd0 2024-09-10T18:37:29.446 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3074117f10 con 0x7f3074100bd0 2024-09-10T18:37:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f3068001e30 con 0x7f3074100bd0 2024-09-10T18:37:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.433+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3068003040 con 0x7f3074100bd0 2024-09-10T18:37:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.434+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3054005040 con 0x7f3074100bd0 2024-09-10T18:37:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.435+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f30680019a0 con 0x7f3074100bd0 2024-09-10T18:37:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.435+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(38..38 src has 1..38) v4 ==== 3493+0+0 (unknown 2772159038 0 0) 0x7f3068086860 con 0x7f3074100bd0 2024-09-10T18:37:29.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.440+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f3068053fc0 con 0x7f3074100bd0 2024-09-10T18:37:29.605 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:29.604+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3054000bc0 con 0x7f305c06fc80 2024-09-10T18:37:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:29 smithi175 ceph-mon[33399]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:29 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:29 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:29 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[40885]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[29369]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:29 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:30.966 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:30 smithi175 ceph-mon[33399]: from='client.34283 v1:172.21.15.175:0/3870903684' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:30 smithi131 ceph-mon[29369]: from='client.34283 v1:172.21.15.175:0/3870903684' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:31.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:30 smithi131 ceph-mon[40885]: from='client.34283 v1:172.21.15.175:0/3870903684' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:31 smithi175 ceph-mon[33399]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:31 smithi131 ceph-mon[29369]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:32.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:31 smithi131 ceph-mon[40885]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/1426790028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]: dispatch 2024-09-10T18:37:33.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:32 smithi175 ceph-mon[33399]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]: dispatch 2024-09-10T18:37:33.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:32 smithi175 ceph-mon[33399]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]': finished 2024-09-10T18:37:33.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:32 smithi175 ceph-mon[33399]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:37:33.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:32 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:33.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/2162947548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/1426790028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]: dispatch 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[29369]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]: dispatch 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[29369]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]': finished 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[29369]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/2162947548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:33.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/1426790028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]: dispatch 2024-09-10T18:37:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[40885]: from='client.24292 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]: dispatch 2024-09-10T18:37:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[40885]: from='client.24292 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8d8b5fe9-ef7d-4b85-a61b-343a3e69a751"}]': finished 2024-09-10T18:37:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[40885]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:37:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/2162947548' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:34.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:33 smithi175 ceph-mon[33399]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:34.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:33 smithi131 ceph-mon[29369]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:34.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:33 smithi131 ceph-mon[40885]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:36.027 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:35 smithi175 ceph-mon[33399]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:36.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:35 smithi131 ceph-mon[29369]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:36.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:35 smithi131 ceph-mon[40885]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:37 smithi175 ceph-mon[33399]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:38.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:37 smithi131 ceph-mon[29369]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:37 smithi131 ceph-mon[40885]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:38 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:37:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:38 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:39.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:38 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:37:39.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:38 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:38 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:37:39.367 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:38 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:40.037 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:39 smithi175 ceph-mon[33399]: Deploying daemon osd.6 on smithi175 2024-09-10T18:37:40.037 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:39 smithi175 ceph-mon[33399]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:39 smithi131 ceph-mon[29369]: Deploying daemon osd.6 on smithi175 2024-09-10T18:37:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:39 smithi131 ceph-mon[29369]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:40.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:39 smithi131 ceph-mon[40885]: Deploying daemon osd.6 on smithi175 2024-09-10T18:37:40.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:39 smithi131 ceph-mon[40885]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:41 smithi131 ceph-mon[29369]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:42.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:41 smithi131 ceph-mon[40885]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:42.414 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:41 smithi175 ceph-mon[33399]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:42 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:42 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:42 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:42 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:42 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:42 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:43.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:42 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:43.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:42 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:43.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:42 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:44.208 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:43 smithi175 ceph-mon[33399]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:44.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:43 smithi131 ceph-mon[29369]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:44.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:43 smithi131 ceph-mon[40885]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:46.107 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:45 smithi175 ceph-mon[33399]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:46.107 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:45 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:46.107 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:45 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:45 smithi131 ceph-mon[29369]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:45 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:45 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:46.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:45 smithi131 ceph-mon[40885]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:46.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:45 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:46.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:45 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:46.537 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 6 on host 'smithi175' 2024-09-10T18:37:46.537 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.535+0000 7f3062ffd640 1 -- 172.21.15.175:0/3870903684 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3415152442) 0x7f3054000bc0 con 0x7f305c06fc80 2024-09-10T18:37:46.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.537+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 >> v1:172.21.15.131:6800/3123609852 conn(0x7f305c06fc80 legacy=0x7f305c072130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:46.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.537+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 >> v1:172.21.15.131:6790/0 conn(0x7f3074100bd0 legacy=0x7f3074112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:46.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.537+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 shutdown_connections 2024-09-10T18:37:46.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.537+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 >> 172.21.15.175:0/3870903684 conn(0x7f30740f63f0 msgr2=0x7f30740f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:46.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.538+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 shutdown_connections 2024-09-10T18:37:46.539 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:46.538+0000 7f3078bb2640 1 -- 172.21.15.175:0/3870903684 wait complete. 2024-09-10T18:37:47.098 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='osd.6 v1:172.21.15.175:6808/3246498802' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:37:47.098 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:37:47.098 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:47.098 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:47.098 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.098 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:47.099 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.099 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:46 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.179 DEBUG:teuthology.orchestra.run.smithi175:osd.6> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:37:47.181 INFO:tasks.cephadm:Deploying osd.7 on smithi175 with /dev/vg_nvme/lv_1... 2024-09-10T18:37:47.182 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='osd.6 v1:172.21.15.175:6808/3246498802' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='osd.6 v1:172.21.15.175:6808/3246498802' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:46 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:47.899 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:37:48.279 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-mon[33399]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:48.279 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-mon[33399]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:37:48.279 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-mon[33399]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:37:48.279 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:48.280 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-mon[33399]: from='osd.6 v1:172.21.15.175:6808/3246498802' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:48.280 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-mon[33399]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:48.280 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:37:47.956+0000 7fa23d0d9640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T18:37:48.280 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:37:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:37:47.961+0000 7fa23670d640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:37:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[29369]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[29369]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:37:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[29369]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:37:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[29369]: from='osd.6 v1:172.21.15.175:6808/3246498802' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[29369]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[40885]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[40885]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[40885]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[40885]: from='osd.6 v1:172.21.15.175:6808/3246498802' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:47 smithi131 ceph-mon[40885]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:37:49.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:48 smithi175 ceph-mon[33399]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:49.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:48 smithi175 ceph-mon[33399]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:37:49.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:48 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:49.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:48 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[29369]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[29369]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:37:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:49.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[40885]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:37:49.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[40885]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:37:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:48 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:37:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:37:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: osd.6 v1:172.21.15.175:6808/3246498802 boot 2024-09-10T18:37:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:37:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:37:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:49 smithi175 ceph-mon[33399]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:37:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:37:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:37:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: osd.6 v1:172.21.15.175:6808/3246498802 boot 2024-09-10T18:37:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[29369]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: osd.6 v1:172.21.15.175:6808/3246498802 boot 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:37:50.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:49 smithi131 ceph-mon[40885]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:37:50.335 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:37:50.438 INFO:teuthology.orchestra.run.smithi175.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:37:50.453 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10+0 records in 2024-09-10T18:37:50.453 INFO:teuthology.orchestra.run.smithi175.stderr:10+0 records out 2024-09-10T18:37:50.453 INFO:teuthology.orchestra.run.smithi175.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135482 s, 774 MB/s 2024-09-10T18:37:50.453 INFO:teuthology.orchestra.run.smithi175.stderr:--> Zapping successful for: 2024-09-10T18:37:51.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi175:vg_nvme/lv_1 2024-09-10T18:37:52.065 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:37:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:37:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:37:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:52.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:52.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:51 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:51 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:37:53.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:52 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi175 2024-09-10T18:37:53.137 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:52 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-10T18:37:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:52 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi175 2024-09-10T18:37:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:52 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-10T18:37:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:52 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi175 2024-09-10T18:37:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:52 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi175 to 5696M 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.642+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3909201496 >> v1:172.21.15.175:6789/0 conn(0x7f2fc00ff990 legacy=0x7f2fc0101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.642+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3909201496 shutdown_connections 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.642+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3909201496 >> 172.21.15.175:0/3909201496 conn(0x7f2fc00f6c30 msgr2=0x7f2fc00f9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.642+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3909201496 shutdown_connections 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.642+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3909201496 wait complete. 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc7e11640 1 Processor -- start 2024-09-10T18:37:53.649 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc7e11640 1 -- start start 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc7e11640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2fc010d1f0 con 0x7f2fc01035b0 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc7e11640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2fc01189f0 con 0x7f2fc00ff990 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc7e11640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2fc0119b90 con 0x7f2fc00fb490 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc53ac640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f2fc00ff990 0x7f2fc0112b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:42040/0 (socket says 172.21.15.175:42040) 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fc53ac640 1 -- 172.21.15.175:0/3433240849 learned_addr learned my addr 172.21.15.175:0/3433240849 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285475093 0 0) 0x7f2fc01189f0 con 0x7f2fc00ff990 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fa8003120 con 0x7f2fc00ff990 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3558651495 0 0) 0x7f2fc0119b90 con 0x7f2fc00fb490 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc01189f0 con 0x7f2fc00fb490 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959315128 0 0) 0x7f2fc010d1f0 con 0x7f2fc01035b0 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fc0119b90 con 0x7f2fc01035b0 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1779480858 0 0) 0x7f2fa8003120 con 0x7f2fc00ff990 2024-09-10T18:37:53.650 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2fc010d1f0 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.643+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f2fb4002f90 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4192981127 0 0) 0x7f2fc010d1f0 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 >> v1:172.21.15.131:6790/0 conn(0x7f2fc00fb490 legacy=0x7f2fc010c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 >> v1:172.21.15.131:6789/0 conn(0x7f2fc01035b0 legacy=0x7f2fc01162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc011ad70 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc0119d80 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc011a1d0 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f2fb4001b70 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.644+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f2fb40038a0 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.645+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fac005040 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.645+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f2fb4003ac0 con 0x7f2fc00ff990 2024-09-10T18:37:53.651 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.646+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(44..44 src has 1..44) v4 ==== 3785+0+0 (unknown 2007224342 0 0) 0x7f2fb4089890 con 0x7f2fc00ff990 2024-09-10T18:37:53.652 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.650+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2fb4056f20 con 0x7f2fc00ff990 2024-09-10T18:37:53.814 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:37:53.812+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fac000bc0 con 0x7f2fa806f930 2024-09-10T18:37:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:53 smithi175 ceph-mon[33399]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:37:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:53 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:53 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:53 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:54.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[29369]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:37:54.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:54.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:54.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:54.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[40885]: pgmap v149: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:37:54.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:54.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:54.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:53 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:55.162 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:54 smithi175 ceph-mon[33399]: from='client.24313 v1:172.21.15.175:0/3433240849' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:55.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:54 smithi131 ceph-mon[29369]: from='client.24313 v1:172.21.15.175:0/3433240849' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:55.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:54 smithi131 ceph-mon[40885]: from='client.24313 v1:172.21.15.175:0/3433240849' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi175:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:55 smithi175 ceph-mon[33399]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:37:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:55 smithi175 ceph-mon[33399]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:37:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:55 smithi175 ceph-mon[33399]: Cluster is now healthy 2024-09-10T18:37:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:55 smithi131 ceph-mon[29369]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:37:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:55 smithi131 ceph-mon[29369]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:37:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:55 smithi131 ceph-mon[29369]: Cluster is now healthy 2024-09-10T18:37:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:55 smithi131 ceph-mon[40885]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:37:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:55 smithi131 ceph-mon[40885]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:37:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:55 smithi131 ceph-mon[40885]: Cluster is now healthy 2024-09-10T18:37:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/1097990261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]: dispatch 2024-09-10T18:37:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[29369]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]: dispatch 2024-09-10T18:37:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[29369]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]': finished 2024-09-10T18:37:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[29369]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:37:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:37:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/152416744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/1097990261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]: dispatch 2024-09-10T18:37:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[40885]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]: dispatch 2024-09-10T18:37:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[40885]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]': finished 2024-09-10T18:37:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[40885]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:37:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:37:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/152416744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/1097990261' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]: dispatch 2024-09-10T18:37:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:56 smithi175 ceph-mon[33399]: from='client.24319 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]: dispatch 2024-09-10T18:37:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:56 smithi175 ceph-mon[33399]: from='client.24319 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbba104c-8b89-4955-8189-3aa160932f1b"}]': finished 2024-09-10T18:37:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:56 smithi175 ceph-mon[33399]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:37:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:56 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:37:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/152416744' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:58.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:57 smithi131 ceph-mon[29369]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T18:37:58.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:57 smithi131 ceph-mon[40885]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T18:37:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:57 smithi175 ceph-mon[33399]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-10T18:38:00.315 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:37:59 smithi175 ceph-mon[33399]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:38:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:37:59 smithi131 ceph-mon[29369]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:38:00.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:37:59 smithi131 ceph-mon[40885]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:38:02.316 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:01 smithi175 ceph-mon[33399]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T18:38:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:01 smithi131 ceph-mon[29369]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T18:38:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:01 smithi131 ceph-mon[40885]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T18:38:03.027 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:02 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:38:03.027 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:02 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:38:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:02 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:02 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:38:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:02 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:04.050 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:03 smithi175 ceph-mon[33399]: Deploying daemon osd.7 on smithi175 2024-09-10T18:38:04.050 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:03 smithi175 ceph-mon[33399]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:38:04.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:03 smithi131 ceph-mon[29369]: Deploying daemon osd.7 on smithi175 2024-09-10T18:38:04.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:03 smithi131 ceph-mon[29369]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:38:04.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:03 smithi131 ceph-mon[40885]: Deploying daemon osd.7 on smithi175 2024-09-10T18:38:04.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:03 smithi131 ceph-mon[40885]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:38:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:05 smithi131 ceph-mon[29369]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:05 smithi131 ceph-mon[40885]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:06.423 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:05 smithi175 ceph-mon[33399]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:08.108 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:07 smithi175 ceph-mon[33399]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:08.108 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:08.108 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:08.108 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:07 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[29369]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[40885]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:08.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:08.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:07 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:10.271 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:09 smithi175 ceph-mon[33399]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:10.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:09 smithi131 ceph-mon[29369]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:10.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:09 smithi131 ceph-mon[40885]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:10.588 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.586+0000 7f2fbe7fc640 1 -- 172.21.15.175:0/3433240849 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2543516046) 0x7f2fac000bc0 con 0x7f2fa806f930 2024-09-10T18:38:10.589 INFO:teuthology.orchestra.run.smithi175.stdout:Created osd(s) 7 on host 'smithi175' 2024-09-10T18:38:10.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.588+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 >> v1:172.21.15.131:6800/3123609852 conn(0x7f2fa806f930 legacy=0x7f2fa8071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:10.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.589+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 >> v1:172.21.15.175:6789/0 conn(0x7f2fc00ff990 legacy=0x7f2fc0112b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:10.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.589+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 shutdown_connections 2024-09-10T18:38:10.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.589+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 >> 172.21.15.175:0/3433240849 conn(0x7f2fc00f6c30 msgr2=0x7f2fc01068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:10.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.589+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 shutdown_connections 2024-09-10T18:38:10.591 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:10.589+0000 7f2fc7e11640 1 -- 172.21.15.175:0/3433240849 wait complete. 2024-09-10T18:38:11.199 DEBUG:teuthology.orchestra.run.smithi175:osd.7> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:38:11.202 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:38:11.202 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:38:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:10 smithi175 ceph-mon[33399]: from='osd.7 v1:172.21.15.175:6812/2356238523' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:38:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:10 smithi175 ceph-mon[33399]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:38:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:10 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:11.224 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:10 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[29369]: from='osd.7 v1:172.21.15.175:6812/2356238523' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:38:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[29369]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:38:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[40885]: from='osd.7 v1:172.21.15.175:6812/2356238523' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:38:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[40885]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:38:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:10 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:11.462 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:38:12.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:12.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:38:12.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:38:12.283 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: from='osd.7 v1:172.21.15.175:6812/2356238523' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: from='osd.7 v1:172.21.15.175:6812/2356238523' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:12.284 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:11 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: from='osd.7 v1:172.21.15.175:6812/2356238523' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]: dispatch 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:12.388 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:11 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:12.389 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:38:12 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:38:12.005+0000 7fa836881640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T18:38:12.389 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:38:12 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:38:12.011+0000 7fa82feb5640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:38:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:38:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:38:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: osd.7 v1:172.21.15.175:6812/2356238523 boot 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: osd.7 v1:172.21.15.175:6812/2356238523 boot 2024-09-10T18:38:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:38:13.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:13 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.317 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1382874483 >> v1:172.21.15.131:6789/0 conn(0x7fbcac1046b0 legacy=0x7fbcac106b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1382874483 shutdown_connections 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1382874483 >> 172.21.15.131:0/1382874483 conn(0x7fbcac0f7d30 msgr2=0x7fbcac0fa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1382874483 shutdown_connections 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1382874483 wait complete. 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 Processor -- start 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.315+0000 7fbcb2e2c640 1 -- start start 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb2e2c640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbcac10d120 con 0x7fbcac0fc590 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb2e2c640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbcac118920 con 0x7fbcac1046b0 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb2e2c640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbcac119ac0 con 0x7fbcac100a90 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb0bc8640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fbcac0fc590 0x7fbcac10c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39884/0 (socket says 172.21.15.131:39884) 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb0bc8640 1 -- 172.21.15.131:0/1083359095 learned_addr learned my addr 172.21.15.131:0/1083359095 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcabfff640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fbcac100a90 0x7fbcac112ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:35118/0 (socket says 172.21.15.131:35118) 2024-09-10T18:38:13.318 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3958316172 0 0) 0x7fbcac119ac0 con 0x7fbcac100a90 2024-09-10T18:38:13.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc9c0034e0 con 0x7fbcac100a90 2024-09-10T18:38:13.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542782349 0 0) 0x7fbcac10d120 con 0x7fbcac0fc590 2024-09-10T18:38:13.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcac119ac0 con 0x7fbcac0fc590 2024-09-10T18:38:13.320 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811272762 0 0) 0x7fbcac118920 con 0x7fbcac1046b0 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcac10d120 con 0x7fbcac1046b0 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 827747821 0 0) 0x7fbc9c0034e0 con 0x7fbcac100a90 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbcac118920 con 0x7fbcac100a90 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2125402488 0 0) 0x7fbcac119ac0 con 0x7fbcac0fc590 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc9c0034e0 con 0x7fbcac0fc590 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fbc980019f0 con 0x7fbcac100a90 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1175662025 0 0) 0x7fbcac10d120 con 0x7fbcac1046b0 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbcac119ac0 con 0x7fbcac1046b0 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fbca0001880 con 0x7fbcac0fc590 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fbca4002130 con 0x7fbcac1046b0 2024-09-10T18:38:13.321 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3922656139 0 0) 0x7fbcac118920 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 >> v1:172.21.15.175:6789/0 conn(0x7fbcac1046b0 legacy=0x7fbcac1161f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 >> v1:172.21.15.131:6789/0 conn(0x7fbcac0fc590 legacy=0x7fbcac10c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac11aca0 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac117970 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.316+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac117e40 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.317+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fbc98002310 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.317+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fbc980058a0 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.318+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc90005040 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.318+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7fbc980050b0 con 0x7fbcac100a90 2024-09-10T18:38:13.322 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.319+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(48..48 src has 1..48) v4 ==== 4110+0+0 (unknown 606182974 0 0) 0x7fbc98089390 con 0x7fbcac100a90 2024-09-10T18:38:13.324 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.323+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fbc980568e0 con 0x7fbcac100a90 2024-09-10T18:38:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi175", "root=default"]}]': finished 2024-09-10T18:38:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:38:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:38:13.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: osd.7 v1:172.21.15.175:6812/2356238523 boot 2024-09-10T18:38:13.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:38:13.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:13 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:38:13.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.480+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fbc900052f0 con 0x7fbcac100a90 2024-09-10T18:38:13.482 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.480+0000 7fbca97fa640 1 -- 172.21.15.131:0/1083359095 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v48) v1 ==== 74+0+130 (unknown 4182864030 0 2236976402) 0x7fbc98056280 con 0x7fbcac100a90 2024-09-10T18:38:13.482 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:38:13.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.483+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 >> v1:172.21.15.131:6800/3123609852 conn(0x7fbc9c06fe20 legacy=0x7fbc9c0722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:13.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.483+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 >> v1:172.21.15.131:6790/0 conn(0x7fbcac100a90 legacy=0x7fbcac112ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:13.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.483+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 shutdown_connections 2024-09-10T18:38:13.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.483+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 >> 172.21.15.131:0/1083359095 conn(0x7fbcac0f7d30 msgr2=0x7fbcac106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:13.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.483+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 shutdown_connections 2024-09-10T18:38:13.485 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:13.483+0000 7fbcb2e2c640 1 -- 172.21.15.131:0/1083359095 wait complete. 2024-09-10T18:38:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[29369]: purged_snaps scrub starts 2024-09-10T18:38:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[29369]: purged_snaps scrub ok 2024-09-10T18:38:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1083359095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:38:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[29369]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:38:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[40885]: purged_snaps scrub starts 2024-09-10T18:38:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[40885]: purged_snaps scrub ok 2024-09-10T18:38:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1083359095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:38:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:14 smithi131 ceph-mon[40885]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:38:14.128 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1725993492,"num_in_osds":8,"osd_in_since":1725993476,"num_remapped_pgs":1} 2024-09-10T18:38:14.129 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:38:14.362 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:14 smithi175 ceph-mon[33399]: purged_snaps scrub starts 2024-09-10T18:38:14.362 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:14 smithi175 ceph-mon[33399]: purged_snaps scrub ok 2024-09-10T18:38:14.362 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1083359095' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:38:14.362 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:14 smithi175 ceph-mon[33399]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:38:14.372 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:38:15.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: Detected new or changed devices on smithi175 2024-09-10T18:38:15.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.525 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:15.526 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[40885]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: Detected new or changed devices on smithi175 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.527 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.528 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:15 smithi131 ceph-mon[29369]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:38:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: Detected new or changed devices on smithi175 2024-09-10T18:38:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi175 to 4272M 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:15.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:15.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:15 smithi175 ceph-mon[33399]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:38:15.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 -- 172.21.15.131:0/1888169181 >> v1:172.21.15.131:6789/0 conn(0x7fcde8105340 legacy=0x7fcde8107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:15.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 -- 172.21.15.131:0/1888169181 shutdown_connections 2024-09-10T18:38:15.978 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 -- 172.21.15.131:0/1888169181 >> 172.21.15.131:0/1888169181 conn(0x7fcde80f8140 msgr2=0x7fcde80fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 -- 172.21.15.131:0/1888169181 shutdown_connections 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 -- 172.21.15.131:0/1888169181 wait complete. 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 Processor -- start 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.969+0000 7fcdeed19640 1 -- start start 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcdeed19640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcde8101af0 con 0x7fcde8102580 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcdeed19640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcde8101ca0 con 0x7fcde8105340 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcdeed19640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcde8101e90 con 0x7fcde8108ed0 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcdecab5640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fcde8102580 0x7fcde8101350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:39896/0 (socket says 172.21.15.131:39896) 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcdecab5640 1 -- 172.21.15.131:0/73351991 learned_addr learned my addr 172.21.15.131:0/73351991 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794360874 0 0) 0x7fcde8101e90 con 0x7fcde8108ed0 2024-09-10T18:38:15.979 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd80034e0 con 0x7fcde8108ed0 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266305512 0 0) 0x7fcde8101af0 con 0x7fcde8102580 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde8101e90 con 0x7fcde8102580 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211075250 0 0) 0x7fcde8101ca0 con 0x7fcde8105340 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde8101af0 con 0x7fcde8105340 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 566750395 0 0) 0x7fcdd80034e0 con 0x7fcde8108ed0 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcde8101ca0 con 0x7fcde8108ed0 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fcde0001d50 con 0x7fcde8108ed0 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1390400507 0 0) 0x7fcde8101e90 con 0x7fcde8102580 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcdd80034e0 con 0x7fcde8102580 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.970+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3400337818 0 0) 0x7fcde8101af0 con 0x7fcde8105340 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcde8101e90 con 0x7fcde8105340 2024-09-10T18:38:15.980 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fcddc001900 con 0x7fcde8102580 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fcdd4001500 con 0x7fcde8105340 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3035797966 0 0) 0x7fcde8101ca0 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 >> v1:172.21.15.175:6789/0 conn(0x7fcde8105340 legacy=0x7fcde81198f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 >> v1:172.21.15.131:6789/0 conn(0x7fcde8102580 legacy=0x7fcde8101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8121770 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde811e7a0 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde811ec70 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fcde0002670 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.971+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fcde0003580 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.972+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdcc005010 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.972+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7fcde0002220 con 0x7fcde8108ed0 2024-09-10T18:38:15.981 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.973+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 3855581764 0 0) 0x7fcde0086ef0 con 0x7fcde8108ed0 2024-09-10T18:38:15.982 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:15.977+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fcde0054440 con 0x7fcde8108ed0 2024-09-10T18:38:16.135 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.134+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdcc0052c0 con 0x7fcde8108ed0 2024-09-10T18:38:16.136 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.134+0000 7fcde57fa640 1 -- 172.21.15.131:0/73351991 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+11198 (unknown 1916093452 0 1714880808) 0x7fcde0053de0 con 0x7fcde8108ed0 2024-09-10T18:38:16.136 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:38:16.137 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":50,"fsid":"0846fc60-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:33:07.857701+0000","modified":"2024-09-10T18:38:14.639336+0000","last_up_change":"2024-09-10T18:38:12.634608+0000","last_in_change":"2024-09-10T18:37:56.112167+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:36:16.640762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6801","nonce":437930310}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6802","nonce":437930310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6804","nonce":437930310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6803","nonce":437930310}]},"public_addr":"172.21.15.131:6801/437930310","cluster_addr":"172.21.15.131:6802/437930310","heartbeat_back_addr":"172.21.15.131:6804/437930310","heartbeat_front_addr":"172.21.15.131:6803/437930310","state":["exists","up"]},{"osd":1,"uuid":"20feda32-c9c6-42f4-8226-1652eccf014e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6805","nonce":2444922616}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6806","nonce":2444922616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6808","nonce":2444922616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6807","nonce":2444922616}]},"public_addr":"172.21.15.131:6805/2444922616","cluster_addr":"172.21.15.131:6806/2444922616","heartbeat_back_addr":"172.21.15.131:6808/2444922616","heartbeat_front_addr":"172.21.15.131:6807/2444922616","state":["exists","up"]},{"osd":2,"uuid":"bd6bffb1-379f-41bc-9600-f7aed3a9c7d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6809","nonce":1436967876}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6810","nonce":1436967876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6812","nonce":1436967876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6811","nonce":1436967876}]},"public_addr":"172.21.15.131:6809/1436967876","cluster_addr":"172.21.15.131:6810/1436967876","heartbeat_back_addr":"172.21.15.131:6812/1436967876","heartbeat_front_addr":"172.21.15.131:6811/1436967876","state":["exists","up"]},{"osd":3,"uuid":"5b888d2e-d94c-4cd8-afad-498c49a22a4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6813","nonce":2850347725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6814","nonce":2850347725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6816","nonce":2850347725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6815","nonce":2850347725}]},"public_addr":"172.21.15.131:6813/2850347725","cluster_addr":"172.21.15.131:6814/2850347725","heartbeat_back_addr":"172.21.15.131:6816/2850347725","heartbeat_front_addr":"172.21.15.131:6815/2850347725","state":["exists","up"]},{"osd":4,"uuid":"638c463c-43c6-4ca5-b221-de408d307972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6800","nonce":2653212041}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6801","nonce":2653212041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6803","nonce":2653212041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6802","nonce":2653212041}]},"public_addr":"172.21.15.175:6800/2653212041","cluster_addr":"172.21.15.175:6801/2653212041","heartbeat_back_addr":"172.21.15.175:6803/2653212041","heartbeat_front_addr":"172.21.15.175:6802/2653212041","state":["exists","up"]},{"osd":5,"uuid":"c07e1e1e-2ad2-4acb-b8d2-0a77b6661604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6804","nonce":210062365}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6805","nonce":210062365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6807","nonce":210062365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6806","nonce":210062365}]},"public_addr":"172.21.15.175:6804/210062365","cluster_addr":"172.21.15.175:6805/210062365","heartbeat_back_addr":"172.21.15.175:6807/210062365","heartbeat_front_addr":"172.21.15.175:6806/210062365","state":["exists","up"]},{"osd":6,"uuid":"8d8b5fe9-ef7d-4b85-a61b-343a3e69a751","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6808","nonce":3246498802}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6809","nonce":3246498802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6811","nonce":3246498802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6810","nonce":3246498802}]},"public_addr":"172.21.15.175:6808/3246498802","cluster_addr":"172.21.15.175:6809/3246498802","heartbeat_back_addr":"172.21.15.175:6811/3246498802","heartbeat_front_addr":"172.21.15.175:6810/3246498802","state":["exists","up"]},{"osd":7,"uuid":"bbba104c-8b89-4955-8189-3aa160932f1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6812","nonce":2356238523}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6813","nonce":2356238523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6815","nonce":2356238523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6814","nonce":2356238523}]},"public_addr":"172.21.15.175:6812/2356238523","cluster_addr":"172.21.15.175:6813/2356238523","heartbeat_back_addr":"172.21.15.175:6815/2356238523","heartbeat_front_addr":"172.21.15.175:6814/2356238523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:35:25.814053+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:35:50.012887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:13.994960+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:37.876356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:59.595337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:23.108539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:47.058002+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:11.527856+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6800/2393239631":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/1393657916":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/3549403922":"2024-09-11T18:34:02.347093+0000","172.21.15.131:6800/1625784054":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/681874383":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/968404841":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/3135886569":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/3002086193":"2024-09-11T18:33:39.624176+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:38:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.136+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 >> v1:172.21.15.131:6800/3123609852 conn(0x7fcdd806fdd0 legacy=0x7fcdd8072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.136+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 >> v1:172.21.15.131:6790/0 conn(0x7fcde8108ed0 legacy=0x7fcde811d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.136+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 shutdown_connections 2024-09-10T18:38:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.137+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 >> 172.21.15.131:0/73351991 conn(0x7fcde80f8140 msgr2=0x7fcde8064c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:16.138 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.137+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 shutdown_connections 2024-09-10T18:38:16.139 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:16.137+0000 7fcdeed19640 1 -- 172.21.15.131:0/73351991 wait complete. 2024-09-10T18:38:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:16 smithi175 ceph-mon[33399]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:38:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/73351991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:38:16.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:16 smithi131 ceph-mon[29369]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:38:16.782 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/73351991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:38:16.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:16 smithi131 ceph-mon[40885]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:38:16.783 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/73351991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:38:16.806 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:36:16.640762+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:38:16.806 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:38:17.046 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:38:17.417 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:17 smithi131 ceph-mon[29369]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:17.418 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:17 smithi131 ceph-mon[40885]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:17 smithi175 ceph-mon[33399]: pgmap v167: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.621+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2318346312 >> v1:172.21.15.131:6789/0 conn(0x7f9a680fe1a0 legacy=0x7f9a68107650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.621+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2318346312 shutdown_connections 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.621+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2318346312 >> 172.21.15.131:0/2318346312 conn(0x7f9a680752c0 msgr2=0x7f9a680f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.621+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2318346312 shutdown_connections 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.621+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2318346312 wait complete. 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a6dc20640 1 Processor -- start 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a6dc20640 1 -- start start 2024-09-10T18:38:18.626 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a6dc20640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a6811be20 con 0x7f9a68108f00 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a6dc20640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a6811cfe0 con 0x7f9a680fd5d0 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a6dc20640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9a6811e1e0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a66ffd640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f9a680fe1a0 0x7f9a68104500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:35162/0 (socket says 172.21.15.131:35162) 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.622+0000 7f9a66ffd640 1 -- 172.21.15.131:0/2747834780 learned_addr learned my addr 172.21.15.131:0/2747834780 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143489888 0 0) 0x7f9a6811be20 con 0x7f9a68108f00 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a500034e0 con 0x7f9a68108f00 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554973868 0 0) 0x7f9a6811e1e0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a6811be20 con 0x7f9a680fe1a0 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818398334 0 0) 0x7f9a6811cfe0 con 0x7f9a680fd5d0 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a6811e1e0 con 0x7f9a680fd5d0 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 820806699 0 0) 0x7f9a500034e0 con 0x7f9a68108f00 2024-09-10T18:38:18.627 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a6811cfe0 con 0x7f9a68108f00 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a60001840 con 0x7f9a68108f00 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2636795438 0 0) 0x7f9a6811be20 con 0x7f9a680fe1a0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a500034e0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 617245895 0 0) 0x7f9a6811e1e0 con 0x7f9a680fd5d0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9a6811be20 con 0x7f9a680fd5d0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a5c0017b0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a58001800 con 0x7f9a680fd5d0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2193109102 0 0) 0x7f9a500034e0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 >> v1:172.21.15.175:6789/0 conn(0x7f9a680fd5d0 legacy=0x7f9a68103df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 >> v1:172.21.15.131:6789/0 conn(0x7f9a68108f00 legacy=0x7f9a6811a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6811f3e0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.628 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6811e3d0 con 0x7f9a680fe1a0 2024-09-10T18:38:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6811e820 con 0x7f9a680fe1a0 2024-09-10T18:38:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f9a5c001500 con 0x7f9a680fe1a0 2024-09-10T18:38:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.623+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9a5c003810 con 0x7f9a680fe1a0 2024-09-10T18:38:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.625+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f9a5c001b40 con 0x7f9a680fe1a0 2024-09-10T18:38:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.625+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54005010 con 0x7f9a680fe1a0 2024-09-10T18:38:18.629 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.626+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 3855581764 0 0) 0x7f9a5c089b60 con 0x7f9a680fe1a0 2024-09-10T18:38:18.631 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.630+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f9a5c057070 con 0x7f9a680fe1a0 2024-09-10T18:38:18.788 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:18 smithi131 ceph-mon[29369]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:38:18.789 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:18 smithi131 ceph-mon[29369]: Cluster is now healthy 2024-09-10T18:38:18.789 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:18 smithi131 ceph-mon[40885]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:38:18.789 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:18 smithi131 ceph-mon[40885]: Cluster is now healthy 2024-09-10T18:38:18.789 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.786+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a54005840 con 0x7f9a680fe1a0 2024-09-10T18:38:18.790 INFO:teuthology.orchestra.run.smithi131.stdout:pg_num: 1 2024-09-10T18:38:18.790 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.786+0000 7f9a4ffff640 1 -- 172.21.15.131:0/2747834780 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (unknown 3046549997 0 2170607528) 0x7f9a5c056a10 con 0x7f9a680fe1a0 2024-09-10T18:38:18.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.789+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 >> v1:172.21.15.131:6800/3123609852 conn(0x7f9a5006fd20 legacy=0x7f9a500721d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:18.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.789+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 >> v1:172.21.15.131:6790/0 conn(0x7f9a680fe1a0 legacy=0x7f9a68104500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:18.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.789+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 shutdown_connections 2024-09-10T18:38:18.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.789+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 >> 172.21.15.131:0/2747834780 conn(0x7f9a680752c0 msgr2=0x7f9a6810c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:18.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.789+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 shutdown_connections 2024-09-10T18:38:18.791 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:18.789+0000 7f9a6dc20640 1 -- 172.21.15.131:0/2747834780 wait complete. 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:18 smithi175 ceph-mon[33399]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:38:18.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:18 smithi175 ceph-mon[33399]: Cluster is now healthy 2024-09-10T18:38:19.505 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi131 2024-09-10T18:38:19.505 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi131=foo.a' 2024-09-10T18:38:19.761 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:19.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:19 smithi131 ceph-mon[29369]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:19.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2747834780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:38:19.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:19 smithi131 ceph-mon[40885]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:19.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2747834780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:38:19.879 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:19 smithi175 ceph-mon[33399]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:19.879 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2747834780' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:38:21.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.345+0000 7fc5841a8640 1 -- 172.21.15.175:0/4281457766 >> v1:172.21.15.175:6789/0 conn(0x7fc57c1035a0 legacy=0x7fc57c105a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:21.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.345+0000 7fc5841a8640 1 -- 172.21.15.175:0/4281457766 shutdown_connections 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.345+0000 7fc5841a8640 1 -- 172.21.15.175:0/4281457766 >> 172.21.15.175:0/4281457766 conn(0x7fc57c0f6c20 msgr2=0x7fc57c0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.345+0000 7fc5841a8640 1 -- 172.21.15.175:0/4281457766 shutdown_connections 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.345+0000 7fc5841a8640 1 -- 172.21.15.175:0/4281457766 wait complete. 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5841a8640 1 Processor -- start 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5841a8640 1 -- start start 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5841a8640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc57c10bd70 con 0x7fc57c1035a0 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5841a8640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc57c117570 con 0x7fc57c0ff980 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5841a8640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc57c118710 con 0x7fc57c0fb480 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc581743640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fc57c0ff980 0x7fc57c111710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:48806/0 (socket says 172.21.15.175:48806) 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc581743640 1 -- 172.21.15.175:0/2056491985 learned_addr learned my addr 172.21.15.175:0/2056491985 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:21.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349949196 0 0) 0x7fc57c117570 con 0x7fc57c0ff980 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc568003120 con 0x7fc57c0ff980 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727067456 0 0) 0x7fc57c118710 con 0x7fc57c0fb480 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c117570 con 0x7fc57c0fb480 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318124212 0 0) 0x7fc57c10bd70 con 0x7fc57c1035a0 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc57c118710 con 0x7fc57c1035a0 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3496568021 0 0) 0x7fc568003120 con 0x7fc57c0ff980 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc57c10bd70 con 0x7fc57c0ff980 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fc56c002900 con 0x7fc57c0ff980 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.346+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 811851909 0 0) 0x7fc57c117570 con 0x7fc57c0fb480 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc568003120 con 0x7fc57c0fb480 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fc578001670 con 0x7fc57c0fb480 2024-09-10T18:38:21.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4110627933 0 0) 0x7fc57c118710 con 0x7fc57c1035a0 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc57c117570 con 0x7fc57c1035a0 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fc574001950 con 0x7fc57c1035a0 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2951467061 0 0) 0x7fc57c10bd70 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 >> v1:172.21.15.131:6790/0 conn(0x7fc57c0fb480 legacy=0x7fc57c10b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 >> v1:172.21.15.131:6789/0 conn(0x7fc57c1035a0 legacy=0x7fc57c114e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c1198f0 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c118900 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c118d50 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fc56c002c90 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.347+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fc56c005a80 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.348+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc560005040 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.349+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7fc56c001180 con 0x7fc57c0ff980 2024-09-10T18:38:21.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.349+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 3855581764 0 0) 0x7fc56c089430 con 0x7fc57c0ff980 2024-09-10T18:38:21.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.353+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fc56c056940 con 0x7fc57c0ff980 2024-09-10T18:38:21.523 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.520+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc560000bc0 con 0x7fc56806fdc0 2024-09-10T18:38:21.530 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.527+0000 7fc5727fc640 1 -- 172.21.15.175:0/2056491985 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7fc560000bc0 con 0x7fc56806fdc0 2024-09-10T18:38:21.531 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:38:21.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.529+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 >> v1:172.21.15.131:6800/3123609852 conn(0x7fc56806fdc0 legacy=0x7fc568072270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:21.532 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.529+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 >> v1:172.21.15.175:6789/0 conn(0x7fc57c0ff980 legacy=0x7fc57c111710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.529+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 shutdown_connections 2024-09-10T18:38:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.529+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 >> 172.21.15.175:0/2056491985 conn(0x7fc57c0f6c20 msgr2=0x7fc57c106a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.529+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 shutdown_connections 2024-09-10T18:38:21.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:21.529+0000 7fc5841a8640 1 -- 172.21.15.175:0/2056491985 wait complete. 2024-09-10T18:38:21.824 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:21 smithi175 ceph-mon[33399]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:21 smithi131 ceph-mon[29369]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:21 smithi131 ceph-mon[40885]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:22.115 DEBUG:teuthology.orchestra.run.smithi131:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:38:22.118 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi175 2024-09-10T18:38:22.119 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:38:22.361 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='client.24358 v1:172.21.15.175:0/2056491985' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-10T18:38:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:22.760 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:22.761 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:22 smithi175 ceph-mon[33399]: Deploying daemon rgw.foo.a on smithi131 2024-09-10T18:38:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='client.24358 v1:172.21.15.175:0/2056491985' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-10T18:38:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[29369]: Deploying daemon rgw.foo.a on smithi131 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='client.24358 v1:172.21.15.175:0/2056491985' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi131=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:22.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:22.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:22 smithi131 ceph-mon[40885]: Deploying daemon rgw.foo.a on smithi131 2024-09-10T18:38:23.753 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:23 smithi131 systemd[1]: Starting Ceph rgw.foo.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:38:23.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.962+0000 7f2196ae7640 1 -- 172.21.15.175:0/5474890 >> v1:172.21.15.175:6789/0 conn(0x7f21900ffa60 legacy=0x7f2190101e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:23.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.962+0000 7f2196ae7640 1 -- 172.21.15.175:0/5474890 shutdown_connections 2024-09-10T18:38:23.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.962+0000 7f2196ae7640 1 -- 172.21.15.175:0/5474890 >> 172.21.15.175:0/5474890 conn(0x7f21900f63c0 msgr2=0x7f21900f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.962+0000 7f2196ae7640 1 -- 172.21.15.175:0/5474890 shutdown_connections 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.962+0000 7f2196ae7640 1 -- 172.21.15.175:0/5474890 wait complete. 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f2196ae7640 1 Processor -- start 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f2196ae7640 1 -- start start 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f2196ae7640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f219010d1a0 con 0x7f21900fcaa0 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f2196ae7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21901189a0 con 0x7f21900ffa60 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f2196ae7640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2190119b40 con 0x7f21901035c0 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f218ffff640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f21900ffa60 0x7f2190112b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:48826/0 (socket says 172.21.15.175:48826) 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.963+0000 7f218ffff640 1 -- 172.21.15.175:0/1049690937 learned_addr learned my addr 172.21.15.175:0/1049690937 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316928640 0 0) 0x7f21901189a0 con 0x7f21900ffa60 2024-09-10T18:38:23.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21800034e0 con 0x7f21900ffa60 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2311361102 0 0) 0x7f219010d1a0 con 0x7f21900fcaa0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21901189a0 con 0x7f21900fcaa0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548159788 0 0) 0x7f2190119b40 con 0x7f21901035c0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f219010d1a0 con 0x7f21901035c0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2900651800 0 0) 0x7f21800034e0 con 0x7f21900ffa60 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2190119b40 con 0x7f21900ffa60 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f217c002d50 con 0x7f21900ffa60 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4176407814 0 0) 0x7f21901189a0 con 0x7f21900fcaa0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21800034e0 con 0x7f21900fcaa0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2891780474 0 0) 0x7f219010d1a0 con 0x7f21901035c0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21901189a0 con 0x7f21901035c0 2024-09-10T18:38:23.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f2184001840 con 0x7f21900fcaa0 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f2188001b40 con 0x7f21901035c0 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3703681881 0 0) 0x7f2190119b40 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 >> v1:172.21.15.131:6790/0 conn(0x7f21901035c0 legacy=0x7f2190116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 >> v1:172.21.15.131:6789/0 conn(0x7f21900fcaa0 legacy=0x7f219010c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219011ad20 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21901179f0 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2190117ec0 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f217c002eb0 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.964+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f217c0036f0 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.965+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2174005040 con 0x7f21900ffa60 2024-09-10T18:38:23.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.966+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f217c003930 con 0x7f21900ffa60 2024-09-10T18:38:23.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.966+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(50..50 src has 1..50) v4 ==== 4077+0+0 (unknown 3855581764 0 0) 0x7f217c089980 con 0x7f21900ffa60 2024-09-10T18:38:23.978 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:23.970+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f217c056ef0 con 0x7f21900ffa60 2024-09-10T18:38:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:23 smithi131 ceph-mon[29369]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T18:38:24.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:23 smithi131 ceph-mon[40885]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T18:38:24.077 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:23 smithi131 podman[69433]: 2024-09-10 18:38:23.862689092 +0000 UTC m=+0.015530130 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:38:24.077 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:23 smithi131 podman[69433]: 2024-09-10 18:38:23.987848481 +0000 UTC m=+0.140689520 container create c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T18:38:24.131 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.128+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f2174005870 con 0x7f21900ffa60 2024-09-10T18:38:24.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:23 smithi175 ceph-mon[33399]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T18:38:24.577 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:24 smithi131 podman[69433]: 2024-09-10 18:38:24.207131352 +0000 UTC m=+0.359972407 container init c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T18:38:24.577 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:24 smithi131 podman[69433]: 2024-09-10 18:38:24.21205882 +0000 UTC m=+0.364899863 container start c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:38:24.577 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:24 smithi131 bash[69433]: c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 2024-09-10T18:38:24.577 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:38:24 smithi131 systemd[1]: Started Ceph rgw.foo.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:38:24.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.911+0000 7f218d7fa640 1 -- 172.21.15.175:0/1049690937 <== mon.1 v1:172.21.15.175:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v51) v1 ==== 160+0+0 (unknown 2978880215 0 0) 0x7f217c056890 con 0x7f21900ffa60 2024-09-10T18:38:24.915 INFO:teuthology.orchestra.run.smithi175.stderr:pool 'datapool' created 2024-09-10T18:38:24.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.915+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 >> v1:172.21.15.131:6800/3123609852 conn(0x7f218006fd30 legacy=0x7f21800721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:24.918 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.915+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 >> v1:172.21.15.175:6789/0 conn(0x7f21900ffa60 legacy=0x7f2190112b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:24.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.915+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 shutdown_connections 2024-09-10T18:38:24.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.915+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 >> 172.21.15.175:0/1049690937 conn(0x7f21900f63c0 msgr2=0x7f21900fac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:24.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.916+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 shutdown_connections 2024-09-10T18:38:24.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:24.916+0000 7f2196ae7640 1 -- 172.21.15.175:0/1049690937 wait complete. 2024-09-10T18:38:25.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/1049690937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:38:25.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:38:25.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:24 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:25.294 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/1049690937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:38:25.294 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:38:25.294 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.294 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.294 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/1049690937' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:25.295 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:24 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:25.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:38:25.789 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:25 smithi175 ceph-mon[33399]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:38:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:25 smithi175 ceph-mon[33399]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-10T18:38:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:25 smithi175 ceph-mon[33399]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:38:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:25 smithi175 ceph-mon[33399]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:38:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:38:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:25 smithi175 ceph-mon[33399]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:38:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[40885]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:38:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[40885]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-10T18:38:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[40885]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:38:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[40885]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:38:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:38:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[40885]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:38:26.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[29369]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:38:26.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[29369]: Saving service rgw.foo.a spec with placement smithi131=foo.a;count:1 2024-09-10T18:38:26.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[29369]: from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:38:26.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[29369]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:38:26.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:38:26.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:25 smithi131 ceph-mon[29369]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:38:27.048 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:26 smithi175 ceph-mon[33399]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:38:27.049 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:26 smithi175 ceph-mon[33399]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:38:27.049 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:27.049 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:27.049 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:27.049 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:26 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[29369]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[29369]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[40885]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:38:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[40885]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:38:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:26 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: Checking dashboard <-> RGW credentials 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: pgmap v174: 36 pgs: 10 creating+peering, 24 unknown, 2 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/1169958578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:38:28.229 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[29369]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: Checking dashboard <-> RGW credentials 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: pgmap v174: 36 pgs: 10 creating+peering, 24 unknown, 2 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:38:28.230 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/1169958578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:38:28.231 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:27 smithi131 ceph-mon[40885]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:38:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: Checking dashboard <-> RGW credentials 2024-09-10T18:38:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: pgmap v174: 36 pgs: 10 creating+peering, 24 unknown, 2 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:38:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:38:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/1169958578' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.24403 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: from='client.24403 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:38:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:27 smithi175 ceph-mon[33399]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:38:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: pgmap v177: 68 pgs: 10 creating+peering, 30 unknown, 28 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T18:38:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:38:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[29369]: Cluster is now healthy 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: pgmap v177: 68 pgs: 10 creating+peering, 30 unknown, 28 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:38:30.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:29 smithi131 ceph-mon[40885]: Cluster is now healthy 2024-09-10T18:38:30.238 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: pgmap v177: 68 pgs: 10 creating+peering, 30 unknown, 28 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1023 B/s wr, 5 op/s 2024-09-10T18:38:30.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:38:30.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2870741583' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: from='client.34361 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:38:30.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:38:30.239 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:29 smithi175 ceph-mon[33399]: Cluster is now healthy 2024-09-10T18:38:30.552 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.175 --placement '1;smithi175=iscsi.a' 2024-09-10T18:38:30.800 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:31.170 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:38:31.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:30 smithi175 ceph-mon[33399]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:38:31.171 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:30 smithi175 ceph-mon[33399]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:38:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:38:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:30 smithi131 ceph-mon[29369]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:38:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:30 smithi131 ceph-mon[29369]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:38:31.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/639652614' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:38:31.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:30 smithi131 ceph-mon[40885]: from='client.34361 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:38:31.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:30 smithi131 ceph-mon[40885]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:38:32.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:31 smithi175 ceph-mon[33399]: pgmap v180: 100 pgs: 18 creating+peering, 41 unknown, 41 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:38:32.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:31 smithi175 ceph-mon[33399]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:38:32.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1983079556' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3583177544' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:31 smithi175 ceph-mon[33399]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.146 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:31 smithi175 ceph-mon[33399]: from='client.24412 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[29369]: pgmap v180: 100 pgs: 18 creating+peering, 41 unknown, 41 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[29369]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1983079556' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3583177544' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[29369]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[29369]: from='client.24412 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[40885]: pgmap v180: 100 pgs: 18 creating+peering, 41 unknown, 41 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:38:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[40885]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:38:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1983079556' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3583177544' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[40885]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:31 smithi131 ceph-mon[40885]: from='client.24412 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:38:32.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.340+0000 7f0c732d0640 1 -- 172.21.15.175:0/1327943343 >> v1:172.21.15.175:6789/0 conn(0x7f0c6c103580 legacy=0x7f0c6c105a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:32.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.340+0000 7f0c732d0640 1 -- 172.21.15.175:0/1327943343 shutdown_connections 2024-09-10T18:38:32.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.340+0000 7f0c732d0640 1 -- 172.21.15.175:0/1327943343 >> 172.21.15.175:0/1327943343 conn(0x7f0c6c0f6c20 msgr2=0x7f0c6c0f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:32.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.340+0000 7f0c732d0640 1 -- 172.21.15.175:0/1327943343 shutdown_connections 2024-09-10T18:38:32.344 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.340+0000 7f0c732d0640 1 -- 172.21.15.175:0/1327943343 wait complete. 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c732d0640 1 Processor -- start 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c732d0640 1 -- start start 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c732d0640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c6c119960 con 0x7f0c6c0fb460 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c732d0640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c6c11ab60 con 0x7f0c6c103580 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c732d0640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c6c11bd60 con 0x7f0c6c0ff960 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c7186d640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0c6c103580 0x7f0c6c118060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:37472/0 (socket says 172.21.15.175:37472) 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c7186d640 1 -- 172.21.15.175:0/547666885 learned_addr learned my addr 172.21.15.175:0/547666885 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3599718453 0 0) 0x7f0c6c11ab60 con 0x7f0c6c103580 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c580034e0 con 0x7f0c6c103580 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129698663 0 0) 0x7f0c6c119960 con 0x7f0c6c0fb460 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c11ab60 con 0x7f0c6c0fb460 2024-09-10T18:38:32.345 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112285617 0 0) 0x7f0c6c11bd60 con 0x7f0c6c0ff960 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.341+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c119960 con 0x7f0c6c0ff960 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4060179332 0 0) 0x7f0c580034e0 con 0x7f0c6c103580 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c6c11bd60 con 0x7f0c6c103580 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0c64002f30 con 0x7f0c6c103580 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3506646907 0 0) 0x7f0c6c11ab60 con 0x7f0c6c0fb460 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c580034e0 con 0x7f0c6c0fb460 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0c680018c0 con 0x7f0c6c0fb460 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 275296629 0 0) 0x7f0c6c119960 con 0x7f0c6c0ff960 2024-09-10T18:38:32.346 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c6c11ab60 con 0x7f0c6c0ff960 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0c5c001840 con 0x7f0c6c0ff960 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3632201864 0 0) 0x7f0c6c11bd60 con 0x7f0c6c103580 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 >> v1:172.21.15.131:6790/0 conn(0x7f0c6c0ff960 legacy=0x7f0c6c1147a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 >> v1:172.21.15.131:6789/0 conn(0x7f0c6c0fb460 legacy=0x7f0c6c1053a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c11cf60 con 0x7f0c6c103580 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c119b50 con 0x7f0c6c103580 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0c64001eb0 con 0x7f0c6c103580 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0c64003bb0 con 0x7f0c6c103580 2024-09-10T18:38:32.347 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.342+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c11a020 con 0x7f0c6c103580 2024-09-10T18:38:32.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.343+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c04bf60 con 0x7f0c6c103580 2024-09-10T18:38:32.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.343+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7f0c64003e40 con 0x7f0c6c103580 2024-09-10T18:38:32.348 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.344+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(58..58 src has 1..58) v4 ==== 5922+0+0 (unknown 191529180 0 0) 0x7f0c6408a430 con 0x7f0c6c103580 2024-09-10T18:38:32.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.348+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0c64057270 con 0x7f0c6c103580 2024-09-10T18:38:32.516 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.514+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c6c0f81e0 con 0x7f0c5806fcf0 2024-09-10T18:38:32.524 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.522+0000 7f0c61ffb640 1 -- 172.21.15.175:0/547666885 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (unknown 0 0 803663096) 0x7f0c6c0f81e0 con 0x7f0c5806fcf0 2024-09-10T18:38:32.524 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:38:32.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.524+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 >> v1:172.21.15.131:6800/3123609852 conn(0x7f0c5806fcf0 legacy=0x7f0c580721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:32.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.524+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 >> v1:172.21.15.175:6789/0 conn(0x7f0c6c103580 legacy=0x7f0c6c118060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:32.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.524+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 shutdown_connections 2024-09-10T18:38:32.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.524+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 >> 172.21.15.175:0/547666885 conn(0x7f0c6c0f6c20 msgr2=0x7f0c6c065400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:32.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.524+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 shutdown_connections 2024-09-10T18:38:32.526 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:32.524+0000 7f0c732d0640 1 -- 172.21.15.175:0/547666885 wait complete. 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='client.24412 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3583177544' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='client.24412 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1983079556' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.143 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.144 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:32 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:33.169 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:38:33.169 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:38:33.169 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:38:33.201 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:38:33.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:38:33.229 DEBUG:teuthology.orchestra.run.smithi175:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:38:33.273 INFO:tasks.cephadm:Adding prometheus.a on smithi175 2024-09-10T18:38:33.274 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi175=a' 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='client.24412 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3583177544' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='client.24412 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1983079556' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='client.24412 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3583177544' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='client.24412 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1983079556' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='client.34403 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:38:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:32 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: pgmap v183: 132 pgs: 10 creating+peering, 41 unknown, 81 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:38:34.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='client.24421 v1:172.21.15.175:0/547666885' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:34.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-10T18:38:34.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='client.24412 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:38:34.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:33 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:34.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: pgmap v183: 132 pgs: 10 creating+peering, 41 unknown, 81 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:38:34.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='client.24421 v1:172.21.15.175:0/547666885' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='client.24412 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:34.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: pgmap v183: 132 pgs: 10 creating+peering, 41 unknown, 81 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='client.24421 v1:172.21.15.175:0/547666885' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.175", "placement": "1;smithi175=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: Saving service iscsi.datapool spec with placement smithi175=iscsi.a;count:1 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='client.24412 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='client.34403 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:38:34.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:33 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:34.597 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:35.011 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:34 smithi175 ceph-mon[33399]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-10T18:38:35.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:34 smithi131 ceph-mon[29369]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-10T18:38:35.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:34 smithi131 ceph-mon[40885]: Deploying daemon iscsi.iscsi.a on smithi175 2024-09-10T18:38:36.086 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:35 smithi175 ceph-mon[33399]: pgmap v185: 132 pgs: 7 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 3.4 KiB/s wr, 71 op/s 2024-09-10T18:38:36.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:35 smithi131 ceph-mon[29369]: pgmap v185: 132 pgs: 7 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 3.4 KiB/s wr, 71 op/s 2024-09-10T18:38:36.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:35 smithi131 ceph-mon[40885]: pgmap v185: 132 pgs: 7 creating+peering, 16 unknown, 109 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 3.4 KiB/s wr, 71 op/s 2024-09-10T18:38:36.770 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.740+0000 7fb5b8245640 1 -- 172.21.15.175:0/2075087521 >> v1:172.21.15.175:6789/0 conn(0x7fb5b00fb460 legacy=0x7fb5b00fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:36.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.740+0000 7fb5b8245640 1 -- 172.21.15.175:0/2075087521 shutdown_connections 2024-09-10T18:38:36.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.740+0000 7fb5b8245640 1 -- 172.21.15.175:0/2075087521 >> 172.21.15.175:0/2075087521 conn(0x7fb5b00f6c20 msgr2=0x7fb5b00f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:36.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.740+0000 7fb5b8245640 1 -- 172.21.15.175:0/2075087521 shutdown_connections 2024-09-10T18:38:36.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.740+0000 7fb5b8245640 1 -- 172.21.15.175:0/2075087521 wait complete. 2024-09-10T18:38:36.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b8245640 1 Processor -- start 2024-09-10T18:38:36.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b8245640 1 -- start start 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b8245640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb5b010c130 con 0x7fb5b00ff960 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b8245640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb5b0117930 con 0x7fb5b0103580 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b8245640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb5b0118ad0 con 0x7fb5b00fb460 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b67e2640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fb5b0103580 0x7fb5b0115200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:57664/0 (socket says 172.21.15.175:57664) 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5b67e2640 1 -- 172.21.15.175:0/1286043185 learned_addr learned my addr 172.21.15.175:0/1286043185 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871892819 0 0) 0x7fb5b0117930 con 0x7fb5b0103580 2024-09-10T18:38:36.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59c003120 con 0x7fb5b0103580 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847618504 0 0) 0x7fb5b0118ad0 con 0x7fb5b00fb460 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b0117930 con 0x7fb5b00fb460 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001385915 0 0) 0x7fb5b010c130 con 0x7fb5b00ff960 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5b0118ad0 con 0x7fb5b00ff960 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 183252912 0 0) 0x7fb59c003120 con 0x7fb5b0103580 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb5b010c130 con 0x7fb5b0103580 2024-09-10T18:38:36.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.741+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb5a8001c80 con 0x7fb5b0103580 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 329047348 0 0) 0x7fb5b010c130 con 0x7fb5b0103580 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 >> v1:172.21.15.131:6790/0 conn(0x7fb5b00fb460 legacy=0x7fb5b010b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 >> v1:172.21.15.131:6789/0 conn(0x7fb5b00ff960 legacy=0x7fb5b0111ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0119cb0 con 0x7fb5b0103580 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b0118cc0 con 0x7fb5b0103580 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b0119110 con 0x7fb5b0103580 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fb5a8002010 con 0x7fb5b0103580 2024-09-10T18:38:36.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.742+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb5a8005e00 con 0x7fb5b0103580 2024-09-10T18:38:36.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.743+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb594005040 con 0x7fb5b0103580 2024-09-10T18:38:36.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.743+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 15) v1 ==== 90746+0+0 (unknown 2319444954 0 0) 0x7fb5a8002b70 con 0x7fb5b0103580 2024-09-10T18:38:36.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.744+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 5938+0+0 (unknown 3437594943 0 0) 0x7fb5a808a070 con 0x7fb5b0103580 2024-09-10T18:38:36.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.748+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb5a8056e40 con 0x7fb5b0103580 2024-09-10T18:38:36.923 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.922+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb594000bc0 con 0x7fb59c06f900 2024-09-10T18:38:36.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.928+0000 7fb5a27fc640 1 -- 172.21.15.175:0/1286043185 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7fb594000bc0 con 0x7fb59c06f900 2024-09-10T18:38:36.930 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled prometheus update... 2024-09-10T18:38:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.930+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 >> v1:172.21.15.131:6800/3123609852 conn(0x7fb59c06f900 legacy=0x7fb59c071db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.930+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 >> v1:172.21.15.175:6789/0 conn(0x7fb5b0103580 legacy=0x7fb5b0115200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.931+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 shutdown_connections 2024-09-10T18:38:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.931+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 >> 172.21.15.175:0/1286043185 conn(0x7fb5b00f6c20 msgr2=0x7fb5b0105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:36.932 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.931+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 shutdown_connections 2024-09-10T18:38:36.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:36.931+0000 7fb5b8245640 1 -- 172.21.15.175:0/1286043185 wait complete. 2024-09-10T18:38:37.048 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:36 smithi175 podman[63081]: 2024-09-10 18:38:36.961963413 +0000 UTC m=+0.114950071 container create 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T18:38:37.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 podman[63081]: 2024-09-10 18:38:37.407119509 +0000 UTC m=+0.560106165 container init 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T18:38:37.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 podman[63081]: 2024-09-10 18:38:37.412361258 +0000 UTC m=+0.565347914 container start 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T18:38:37.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 bash[63081]: 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 2024-09-10T18:38:37.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 systemd[1]: Started Ceph iscsi.iscsi.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: pgmap v186: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 5.5 KiB/s wr, 154 op/s 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='client.24442 v1:172.21.15.175:0/1286043185' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:37.995 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:38:37.996 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug Started the configuration object watcher 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug Checking for config object changes every 1s 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug Processing osd blocklist entries for this node 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug Configuration does not have an entry for this host(smithi175.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: * Environment: production 2024-09-10T18:38:37.996 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:38:38.045 DEBUG:teuthology.orchestra.run.smithi175:prometheus.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:38:38.047 INFO:tasks.cephadm:Adding node-exporter.a on smithi131 2024-09-10T18:38:38.047 INFO:tasks.cephadm:Adding node-exporter.b on smithi175 2024-09-10T18:38:38.047 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi131=a;smithi175=b' 2024-09-10T18:38:38.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: pgmap v186: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 5.5 KiB/s wr, 154 op/s 2024-09-10T18:38:38.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='client.24442 v1:172.21.15.175:0/1286043185' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:38.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: pgmap v186: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 62 KiB/s rd, 5.5 KiB/s wr, 154 op/s 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='client.24442 v1:172.21.15.175:0/1286043185' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:38.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: Saving service prometheus spec with placement smithi175=a;count:1 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:38:38.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:37 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:38.372 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: Use a production WSGI server instead. 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: * Debug mode: off 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug * Running on all addresses. 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: * Running on all addresses. 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug * Running on http://[fe80::ec4:7aff:fe88:71f1]:5000/ (Press CTRL+C to quit) 2024-09-10T18:38:38.373 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: * Running on http://[fe80::ec4:7aff:fe88:71f1]:5000/ (Press CTRL+C to quit) 2024-09-10T18:38:38.820 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:39.225 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:38 smithi175 ceph-mon[33399]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:38:39.226 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/3524742977' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:38:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:38 smithi131 ceph-mon[29369]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:38:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/3524742977' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:38:39.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:38 smithi131 ceph-mon[40885]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:38:39.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/3524742977' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:38:40.099 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:39 smithi175 ceph-mon[33399]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.9 KiB/s wr, 176 op/s 2024-09-10T18:38:40.099 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:39 smithi175 ceph-mon[33399]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:38:40.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:39 smithi131 ceph-mon[29369]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.9 KiB/s wr, 176 op/s 2024-09-10T18:38:40.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:39 smithi131 ceph-mon[29369]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:38:40.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:39 smithi131 ceph-mon[40885]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s rd, 5.9 KiB/s wr, 176 op/s 2024-09-10T18:38:40.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:39 smithi131 ceph-mon[40885]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:38:40.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- 172.21.15.175:0/2914844209 >> v1:172.21.15.175:6789/0 conn(0x7f82a80fc570 legacy=0x7f82a80fc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:40.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- 172.21.15.175:0/2914844209 shutdown_connections 2024-09-10T18:38:40.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- 172.21.15.175:0/2914844209 >> 172.21.15.175:0/2914844209 conn(0x7f82a80f7d70 msgr2=0x7f82a80fa180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:40.473 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- 172.21.15.175:0/2914844209 shutdown_connections 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- 172.21.15.175:0/2914844209 wait complete. 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 Processor -- start 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- start start 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82a8119820 con 0x7f82a8100a70 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82a811aa20 con 0x7f82a80fc570 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82af9d8640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f82a811bc20 con 0x7f82a8104690 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82ad774640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f82a80fc570 0x7f82a8071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:57752/0 (socket says 172.21.15.175:57752) 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.461+0000 7f82ad774640 1 -- 172.21.15.175:0/3121693263 learned_addr learned my addr 172.21.15.175:0/3121693263 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877987309 0 0) 0x7f82a811aa20 con 0x7f82a80fc570 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8294003720 con 0x7f82a80fc570 2024-09-10T18:38:40.474 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074596043 0 0) 0x7f82a811bc20 con 0x7f82a8104690 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82a811aa20 con 0x7f82a8104690 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4236987508 0 0) 0x7f8294003720 con 0x7f82a80fc570 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f82a811bc20 con 0x7f82a80fc570 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886436661 0 0) 0x7f82a8119820 con 0x7f82a8100a70 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8294003720 con 0x7f82a8100a70 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f82a4002ab0 con 0x7f82a80fc570 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2808341329 0 0) 0x7f82a811bc20 con 0x7f82a80fc570 2024-09-10T18:38:40.475 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 >> v1:172.21.15.131:6790/0 conn(0x7f82a8104690 legacy=0x7f82a8117f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 >> v1:172.21.15.131:6789/0 conn(0x7f82a8100a70 legacy=0x7f82a81145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a811ce20 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a8119a10 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f82a4002cd0 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f82a40039a0 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.462+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a8119e60 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.463+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a804bf60 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.464+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1923808506 0 0) 0x7f82a4003b20 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.464+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1429322001 0 0) 0x7f82a408a250 con 0x7f82a80fc570 2024-09-10T18:38:40.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.468+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f82a4056f80 con 0x7f82a80fc570 2024-09-10T18:38:40.647 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.646+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi175=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f82a811a280 con 0x7f829406fe70 2024-09-10T18:38:40.653 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.651+0000 7f829dffb640 1 -- 172.21.15.175:0/3121693263 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7f82a811a280 con 0x7f829406fe70 2024-09-10T18:38:40.653 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled node-exporter update... 2024-09-10T18:38:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.654+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 >> v1:172.21.15.131:6800/3123609852 conn(0x7f829406fe70 legacy=0x7f8294072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.654+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 >> v1:172.21.15.175:6789/0 conn(0x7f82a80fc570 legacy=0x7f82a8071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.654+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 shutdown_connections 2024-09-10T18:38:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.654+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 >> 172.21.15.175:0/3121693263 conn(0x7f82a80f7d70 msgr2=0x7f82a8104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.654+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 shutdown_connections 2024-09-10T18:38:40.656 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:40.654+0000 7f82af9d8640 1 -- 172.21.15.175:0/3121693263 wait complete. 2024-09-10T18:38:41.436 DEBUG:teuthology.orchestra.run.smithi131:node-exporter.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:38:41.438 DEBUG:teuthology.orchestra.run.smithi175:node-exporter.b> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:38:41.441 INFO:tasks.cephadm:Adding alertmanager.a on smithi131 2024-09-10T18:38:41.441 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi131=a' 2024-09-10T18:38:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:41 smithi175 ceph-mon[33399]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 142 op/s 2024-09-10T18:38:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:41 smithi175 ceph-mon[33399]: from='client.24478 v1:172.21.15.175:0/3121693263' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:41 smithi175 ceph-mon[33399]: Saving service node-exporter spec with placement smithi131=a;smithi175=b;count:2 2024-09-10T18:38:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:41 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[29369]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 142 op/s 2024-09-10T18:38:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[29369]: from='client.24478 v1:172.21.15.175:0/3121693263' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[29369]: Saving service node-exporter spec with placement smithi131=a;smithi175=b;count:2 2024-09-10T18:38:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[40885]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.7 KiB/s wr, 142 op/s 2024-09-10T18:38:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[40885]: from='client.24478 v1:172.21.15.175:0/3121693263' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi131=a;smithi175=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[40885]: Saving service node-exporter spec with placement smithi131=a;smithi175=b;count:2 2024-09-10T18:38:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:41 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:42.194 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: Deploying daemon prometheus.a on smithi175 2024-09-10T18:38:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.0 KiB/s wr, 120 op/s 2024-09-10T18:38:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:43 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: Deploying daemon prometheus.a on smithi175 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.0 KiB/s wr, 120 op/s 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: Deploying daemon prometheus.a on smithi175 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 49 KiB/s rd, 4.0 KiB/s wr, 120 op/s 2024-09-10T18:38:43.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:43 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:43.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.930+0000 7f78d554e640 1 -- 172.21.15.175:0/3936427076 >> v1:172.21.15.175:6789/0 conn(0x7f78d01035a0 legacy=0x7f78d0105a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:43.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.930+0000 7f78d554e640 1 -- 172.21.15.175:0/3936427076 shutdown_connections 2024-09-10T18:38:43.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.930+0000 7f78d554e640 1 -- 172.21.15.175:0/3936427076 >> 172.21.15.175:0/3936427076 conn(0x7f78d00f6c20 msgr2=0x7f78d00f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:43.940 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.930+0000 7f78d554e640 1 -- 172.21.15.175:0/3936427076 shutdown_connections 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.930+0000 7f78d554e640 1 -- 172.21.15.175:0/3936427076 wait complete. 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78d554e640 1 Processor -- start 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78d554e640 1 -- start start 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78d554e640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f78d010d220 con 0x7f78d01035a0 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78d554e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f78d0118a20 con 0x7f78d00fb480 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78d554e640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f78d0119bc0 con 0x7f78d00ff980 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78ceffd640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f78d00fb480 0x7f78d010c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:57760/0 (socket says 172.21.15.175:57760) 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.931+0000 7f78ceffd640 1 -- 172.21.15.175:0/4227423339 learned_addr learned my addr 172.21.15.175:0/4227423339 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163155754 0 0) 0x7f78d0119bc0 con 0x7f78d00ff980 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78b80034e0 con 0x7f78d00ff980 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964164995 0 0) 0x7f78d0118a20 con 0x7f78d00fb480 2024-09-10T18:38:43.941 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d0119bc0 con 0x7f78d00fb480 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2594786831 0 0) 0x7f78d010d220 con 0x7f78d01035a0 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78d0118a20 con 0x7f78d01035a0 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2375984578 0 0) 0x7f78d0119bc0 con 0x7f78d00fb480 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78d010d220 con 0x7f78d00fb480 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3616586259 0 0) 0x7f78b80034e0 con 0x7f78d00ff980 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78d0119bc0 con 0x7f78d00ff980 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78c0001a10 con 0x7f78d00fb480 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78c40018c0 con 0x7f78d00ff980 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2613821720 0 0) 0x7f78d010d220 con 0x7f78d00fb480 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 >> v1:172.21.15.131:6790/0 conn(0x7f78d00ff980 legacy=0x7f78d0112bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 >> v1:172.21.15.131:6789/0 conn(0x7f78d01035a0 legacy=0x7f78d01162f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d011ada0 con 0x7f78d00fb480 2024-09-10T18:38:43.942 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d0119db0 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.932+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d011a200 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.933+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f78c00023c0 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.933+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78c0005af0 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.934+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc005040 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.934+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1923808506 0 0) 0x7f78c001bff0 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.935+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1429322001 0 0) 0x7f78c0089f10 con 0x7f78d00fb480 2024-09-10T18:38:43.943 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:43.939+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f78c00567c0 con 0x7f78d00fb480 2024-09-10T18:38:44.107 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.106+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f78bc000bc0 con 0x7f78b806fcf0 2024-09-10T18:38:44.114 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled alertmanager update... 2024-09-10T18:38:44.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.112+0000 7f78b77fe640 1 -- 172.21.15.175:0/4227423339 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7f78bc000bc0 con 0x7f78b806fcf0 2024-09-10T18:38:44.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.114+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 >> v1:172.21.15.131:6800/3123609852 conn(0x7f78b806fcf0 legacy=0x7f78b80721a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:44.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.114+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 >> v1:172.21.15.175:6789/0 conn(0x7f78d00fb480 legacy=0x7f78d010c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:44.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.114+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 shutdown_connections 2024-09-10T18:38:44.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.114+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 >> 172.21.15.175:0/4227423339 conn(0x7f78d00f6c20 msgr2=0x7f78d0065350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:44.116 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.114+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 shutdown_connections 2024-09-10T18:38:44.117 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:44.114+0000 7f78d554e640 1 -- 172.21.15.175:0/4227423339 wait complete. 2024-09-10T18:38:44.914 DEBUG:teuthology.orchestra.run.smithi131:alertmanager.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:38:44.917 INFO:tasks.cephadm:Adding grafana.a on smithi175 2024-09-10T18:38:44.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi175=a' 2024-09-10T18:38:45.173 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:45.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:45 smithi175 ceph-mon[33399]: from='client.24484 v1:172.21.15.175:0/4227423339' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:45.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:45 smithi175 ceph-mon[33399]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-09-10T18:38:45.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:45 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:45.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:45 smithi175 ceph-mon[33399]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.3 KiB/s wr, 82 op/s 2024-09-10T18:38:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[29369]: from='client.24484 v1:172.21.15.175:0/4227423339' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[29369]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-09-10T18:38:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[29369]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.3 KiB/s wr, 82 op/s 2024-09-10T18:38:45.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[40885]: from='client.24484 v1:172.21.15.175:0/4227423339' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi131=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[40885]: Saving service alertmanager spec with placement smithi131=a;count:1 2024-09-10T18:38:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:45 smithi131 ceph-mon[40885]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 35 KiB/s rd, 2.3 KiB/s wr, 82 op/s 2024-09-10T18:38:47.734 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:47 smithi175 ceph-mon[33399]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 511 B/s wr, 21 op/s 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 -- 172.21.15.175:0/14630529 >> v1:172.21.15.175:6789/0 conn(0x7f57e0100a50 legacy=0x7f57e0102e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 -- 172.21.15.175:0/14630529 shutdown_connections 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 -- 172.21.15.175:0/14630529 >> 172.21.15.175:0/14630529 conn(0x7f57e00f7d30 msgr2=0x7f57e00fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 -- 172.21.15.175:0/14630529 shutdown_connections 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 -- 172.21.15.175:0/14630529 wait complete. 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 Processor -- start 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.733+0000 7f57e5ae7640 1 -- start start 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57e5ae7640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57e0119940 con 0x7f57e0100a50 2024-09-10T18:38:47.771 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57e5ae7640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57e011ab40 con 0x7f57e00fc550 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57e5ae7640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f57e011bd40 con 0x7f57e0104670 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57df7fe640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f57e00fc550 0x7f57e0071320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:46824/0 (socket says 172.21.15.175:46824) 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57df7fe640 1 -- 172.21.15.175:0/3819258612 learned_addr learned my addr 172.21.15.175:0/3819258612 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549707576 0 0) 0x7f57e011ab40 con 0x7f57e00fc550 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57c80034e0 con 0x7f57e00fc550 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070839173 0 0) 0x7f57e0119940 con 0x7f57e0100a50 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e011ab40 con 0x7f57e0100a50 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549812317 0 0) 0x7f57e011bd40 con 0x7f57e0104670 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e0119940 con 0x7f57e0104670 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2007391899 0 0) 0x7f57c80034e0 con 0x7f57e00fc550 2024-09-10T18:38:47.772 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57e011bd40 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f57d0001980 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 296917586 0 0) 0x7f57e011ab40 con 0x7f57e0100a50 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f57c80034e0 con 0x7f57e0100a50 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f57d4001880 con 0x7f57e0100a50 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 482958010 0 0) 0x7f57e011bd40 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 >> v1:172.21.15.131:6790/0 conn(0x7f57e0104670 legacy=0x7f57e0118040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 >> v1:172.21.15.131:6789/0 conn(0x7f57e0100a50 legacy=0x7f57e0114780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e011cf40 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e0119b30 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e011a000 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f57d0002ce0 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.734+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f57d0005b60 con 0x7f57e00fc550 2024-09-10T18:38:47.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.735+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc005040 con 0x7f57e00fc550 2024-09-10T18:38:47.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.736+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1923808506 0 0) 0x7f57d0001f60 con 0x7f57e00fc550 2024-09-10T18:38:47.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.736+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1429322001 0 0) 0x7f57d0089e50 con 0x7f57e00fc550 2024-09-10T18:38:47.774 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.740+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f57d0056720 con 0x7f57e00fc550 2024-09-10T18:38:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:47 smithi131 ceph-mon[29369]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 511 B/s wr, 21 op/s 2024-09-10T18:38:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:47 smithi131 ceph-mon[40885]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 511 B/s wr, 21 op/s 2024-09-10T18:38:47.910 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.908+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 --> v1:172.21.15.131:6800/3123609852 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f57cc000bc0 con 0x7f57c806fa40 2024-09-10T18:38:47.917 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.915+0000 7f57c7fff640 1 -- 172.21.15.175:0/3819258612 <== mgr.14150 v1:172.21.15.131:6800/3123609852 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7f57cc000bc0 con 0x7f57c806fa40 2024-09-10T18:38:47.917 INFO:teuthology.orchestra.run.smithi175.stdout:Scheduled grafana update... 2024-09-10T18:38:47.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.917+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 >> v1:172.21.15.131:6800/3123609852 conn(0x7f57c806fa40 legacy=0x7f57c8071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:47.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.917+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 >> v1:172.21.15.175:6789/0 conn(0x7f57e00fc550 legacy=0x7f57e0071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:47.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.917+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 shutdown_connections 2024-09-10T18:38:47.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.917+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 >> 172.21.15.175:0/3819258612 conn(0x7f57e00f7d30 msgr2=0x7f57e0106b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:47.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.917+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 shutdown_connections 2024-09-10T18:38:47.919 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:47.917+0000 7f57e5ae7640 1 -- 172.21.15.175:0/3819258612 wait complete. 2024-09-10T18:38:48.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:38:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:38:48.936 DEBUG:teuthology.orchestra.run.smithi175:grafana.a> sudo journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:38:48.939 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:38:48.939 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:38:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:48 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:48 smithi175 ceph-mon[33399]: from='client.24490 v1:172.21.15.175:0/3819258612' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:48 smithi175 ceph-mon[33399]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-10T18:38:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:48 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:49.192 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:38:49.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:49.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[29369]: from='client.24490 v1:172.21.15.175:0/3819258612' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:49.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[29369]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-10T18:38:49.313 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:49.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:38:49.313 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[40885]: from='client.24490 v1:172.21.15.175:0/3819258612' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi175=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:49.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[40885]: Saving service grafana spec with placement smithi175=a;count:1 2024-09-10T18:38:49.314 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:48 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:50.060 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:49 smithi131 ceph-mon[29369]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.8 KiB/s rd, 475 B/s wr, 20 op/s 2024-09-10T18:38:50.061 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:49 smithi131 ceph-mon[40885]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.8 KiB/s rd, 475 B/s wr, 20 op/s 2024-09-10T18:38:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:49 smithi175 ceph-mon[33399]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.8 KiB/s rd, 475 B/s wr, 20 op/s 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.015+0000 7fe9a7016640 1 -- 172.21.15.131:0/424210135 >> v1:172.21.15.131:6789/0 conn(0x7fe9a0100890 legacy=0x7fe9a0102d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.015+0000 7fe9a7016640 1 -- 172.21.15.131:0/424210135 shutdown_connections 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.015+0000 7fe9a7016640 1 -- 172.21.15.131:0/424210135 >> 172.21.15.131:0/424210135 conn(0x7fe9a00f6c20 msgr2=0x7fe9a00f9050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.015+0000 7fe9a7016640 1 -- 172.21.15.131:0/424210135 shutdown_connections 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.015+0000 7fe9a7016640 1 -- 172.21.15.131:0/424210135 wait complete. 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe9a7016640 1 Processor -- start 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe9a7016640 1 -- start start 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe9a7016640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe9a010ced0 con 0x7fe9a0100890 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe9a7016640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe9a01186d0 con 0x7fe9a0104670 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe9a7016640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe9a0119870 con 0x7fe9a0065b30 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe99ffff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe9a0100890 0x7fe9a0112870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37700/0 (socket says 172.21.15.131:37700) 2024-09-10T18:38:51.023 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe99ffff640 1 -- 172.21.15.131:0/480741921 learned_addr learned my addr 172.21.15.131:0/480741921 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe9a4db2640 1 --1- 172.21.15.131:0/480741921 >> v1:172.21.15.131:6790/0 conn(0x7fe9a0065b30 0x7fe9a010c340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:50792/0 (socket says 172.21.15.131:50792) 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809553085 0 0) 0x7fe9a010ced0 con 0x7fe9a0100890 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9900034e0 con 0x7fe9a0100890 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3457775425 0 0) 0x7fe9a0119870 con 0x7fe9a0065b30 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.016+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a010ced0 con 0x7fe9a0065b30 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 791756927 0 0) 0x7fe9a01186d0 con 0x7fe9a0104670 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a0119870 con 0x7fe9a0104670 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 638810064 0 0) 0x7fe9900034e0 con 0x7fe9a0100890 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9a01186d0 con 0x7fe9a0100890 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe98c001440 con 0x7fe9a0100890 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4067082985 0 0) 0x7fe9a010ced0 con 0x7fe9a0065b30 2024-09-10T18:38:51.024 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9900034e0 con 0x7fe9a0065b30 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1684531566 0 0) 0x7fe9a0119870 con 0x7fe9a0104670 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe9a010ced0 con 0x7fe9a0104670 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe994001880 con 0x7fe9a0065b30 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe998001bb0 con 0x7fe9a0104670 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2091880230 0 0) 0x7fe9a01186d0 con 0x7fe9a0100890 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 >> v1:172.21.15.131:6790/0 conn(0x7fe9a0065b30 legacy=0x7fe9a010c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 >> v1:172.21.15.175:6789/0 conn(0x7fe9a0104670 legacy=0x7fe9a0115fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a011aa50 con 0x7fe9a0100890 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9a0119a60 con 0x7fe9a0100890 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9a0119f30 con 0x7fe9a0100890 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fe98c0022e0 con 0x7fe9a0100890 2024-09-10T18:38:51.025 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.017+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe98c003660 con 0x7fe9a0100890 2024-09-10T18:38:51.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.018+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe984005040 con 0x7fe9a0100890 2024-09-10T18:38:51.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.019+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1923808506 0 0) 0x7fe98c002450 con 0x7fe9a0100890 2024-09-10T18:38:51.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.019+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1429322001 0 0) 0x7fe98c08a1c0 con 0x7fe9a0100890 2024-09-10T18:38:51.026 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.024+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fe98c056ef0 con 0x7fe9a0100890 2024-09-10T18:38:51.238 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.236+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe984005870 con 0x7fe9a0100890 2024-09-10T18:38:51.241 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.240+0000 7fe99d7fa640 1 -- 172.21.15.131:0/480741921 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 57899324 0 2763373375) 0x7fe98c056890 con 0x7fe9a0100890 2024-09-10T18:38:51.242 INFO:teuthology.orchestra.run.smithi131.stdout:[client.0] 2024-09-10T18:38:51.242 INFO:teuthology.orchestra.run.smithi131.stdout: key = AQA7kuBm2oErDhAAY/Sx/oc+7ewYvq/6vZIw0g== 2024-09-10T18:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.242+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 >> v1:172.21.15.131:6800/3123609852 conn(0x7fe99006fe80 legacy=0x7fe990072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.242+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 >> v1:172.21.15.131:6789/0 conn(0x7fe9a0100890 legacy=0x7fe9a0112870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.242+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 shutdown_connections 2024-09-10T18:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.242+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 >> 172.21.15.131:0/480741921 conn(0x7fe9a00f6c20 msgr2=0x7fe9a0104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.242+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 shutdown_connections 2024-09-10T18:38:51.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:38:51.242+0000 7fe9a7016640 1 -- 172.21.15.131:0/480741921 wait complete. 2024-09-10T18:38:51.876 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:38:51.876 DEBUG:teuthology.orchestra.run.smithi131:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:38:51.876 DEBUG:teuthology.orchestra.run.smithi131:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:38:51.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:38:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:51 smithi175 ceph-mon[33399]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/480741921' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/480741921' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:38:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:51 smithi131 ceph-mon[29369]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/480741921' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:52.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/480741921' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:38:52.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:51 smithi131 ceph-mon[40885]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:52.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/480741921' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:52.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/480741921' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:38:52.709 INFO:teuthology.orchestra.run.smithi175.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:54.102 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:53 smithi175 ceph-mon[33399]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:54.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:53 smithi131 ceph-mon[29369]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:54.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:53 smithi131 ceph-mon[40885]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:55 smithi175 ceph-mon[33399]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:56.178 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 podman[64982]: 2024-09-10 18:38:56.063651416 +0000 UTC m=+0.377220781 container create 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:38:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:55 smithi131 ceph-mon[29369]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:55 smithi131 ceph-mon[40885]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:38:56.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- 172.21.15.175:0/1500530415 >> v1:172.21.15.175:6789/0 conn(0x7fcec00ff980 legacy=0x7fcec0101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:56.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- 172.21.15.175:0/1500530415 shutdown_connections 2024-09-10T18:38:56.356 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- 172.21.15.175:0/1500530415 >> 172.21.15.175:0/1500530415 conn(0x7fcec00f6c40 msgr2=0x7fcec00f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- 172.21.15.175:0/1500530415 shutdown_connections 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- 172.21.15.175:0/1500530415 wait complete. 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 Processor -- start 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- start start 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcec01177d0 con 0x7fcec01035a0 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcec01189d0 con 0x7fcec00fb480 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec81fb640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcec0119bd0 con 0x7fcec00ff980 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.353+0000 7fcec5f97640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fcec00fb480 0x7fcec0066630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:50208/0 (socket says 172.21.15.175:50208) 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fcec5f97640 1 -- 172.21.15.175:0/2434105502 learned_addr learned my addr 172.21.15.175:0/2434105502 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004489512 0 0) 0x7fcec01189d0 con 0x7fcec00fb480 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceac0034e0 con 0x7fcec00fb480 2024-09-10T18:38:56.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594532115 0 0) 0x7fcec01177d0 con 0x7fcec01035a0 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec01189d0 con 0x7fcec01035a0 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600484085 0 0) 0x7fcec0119bd0 con 0x7fcec00ff980 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec01177d0 con 0x7fcec00ff980 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 260549740 0 0) 0x7fceac0034e0 con 0x7fcec00fb480 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcec0119bd0 con 0x7fcec00fb480 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fcebc001940 con 0x7fcec00fb480 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2619431603 0 0) 0x7fcec01189d0 con 0x7fcec01035a0 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fceac0034e0 con 0x7fcec01035a0 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1653227159 0 0) 0x7fcec0119bd0 con 0x7fcec00fb480 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 >> v1:172.21.15.131:6790/0 conn(0x7fcec00ff980 legacy=0x7fcec0112520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 >> v1:172.21.15.131:6789/0 conn(0x7fcec01035a0 legacy=0x7fcec0115ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec011add0 con 0x7fcec00fb480 2024-09-10T18:38:56.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec01179c0 con 0x7fcec00fb480 2024-09-10T18:38:56.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec0117e90 con 0x7fcec00fb480 2024-09-10T18:38:56.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fcebc0023d0 con 0x7fcec00fb480 2024-09-10T18:38:56.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.354+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fcebc005a10 con 0x7fcec00fb480 2024-09-10T18:38:56.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.355+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea4005040 con 0x7fcec00fb480 2024-09-10T18:38:56.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.356+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 16) v1 ==== 90797+0+0 (unknown 1923808506 0 0) 0x7fcebc001f80 con 0x7fcec00fb480 2024-09-10T18:38:56.359 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.356+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1429322001 0 0) 0x7fcebc089e40 con 0x7fcec00fb480 2024-09-10T18:38:56.362 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.360+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fcebc056ab0 con 0x7fcec00fb480 2024-09-10T18:38:56.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.604+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fcea4005870 con 0x7fcec00fb480 2024-09-10T18:38:56.607 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 podman[64982]: 2024-09-10 18:38:56.425115545 +0000 UTC m=+0.738684874 container init 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:38:56.607 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 podman[64982]: 2024-09-10 18:38:56.430294953 +0000 UTC m=+0.743864273 container start 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:38:56.607 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.453Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:38:56.607 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.454Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:38:56.607 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.454Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi175 (none))" 2024-09-10T18:38:56.607 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.454Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.454Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.455Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.455Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.456Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.456Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.459Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:38:56.608 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.459Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.75µs 2024-09-10T18:38:56.609 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.459Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:38:56.609 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.459Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:38:56.609 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.459Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.095µs wal_replay_duration=206.54µs wbl_replay_duration=88ns total_replay_duration=244.011µs 2024-09-10T18:38:56.609 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.460Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:38:56.609 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.460Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:38:56.609 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.460Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:38:56.610 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.475Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.900405ms db_storage=784ns remote_storage=1.143µs web_handler=400ns query_engine=526ns scrape=371.365µs scrape_sd=16.327µs notify=753ns notify_sd=1.615µs rules=14.325004ms tracing=5.523µs 2024-09-10T18:38:56.610 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.475Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:38:56.610 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:38:56.475Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:38:56.610 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 bash[64982]: 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f 2024-09-10T18:38:56.610 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:38:56 smithi175 systemd[1]: Started Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:38:56.612 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.610+0000 7fceb67fc640 1 -- 172.21.15.175:0/2434105502 <== mon.1 v1:172.21.15.175:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (unknown 3508017507 0 2642181262) 0x7fcebc056450 con 0x7fcec00fb480 2024-09-10T18:38:56.612 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-09-10T18:38:56.612 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBAkuBmP3skJBAAwbvgOlhMxrsrzIsBuFyDYQ== 2024-09-10T18:38:56.614 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.612+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 >> v1:172.21.15.131:6800/3123609852 conn(0x7fceac06faf0 legacy=0x7fceac071fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:56.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.612+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 >> v1:172.21.15.175:6789/0 conn(0x7fcec00fb480 legacy=0x7fcec0066630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:38:56.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.612+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 shutdown_connections 2024-09-10T18:38:56.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.612+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 >> 172.21.15.175:0/2434105502 conn(0x7fcec00f6c40 msgr2=0x7fcec0105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:38:56.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.613+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 shutdown_connections 2024-09-10T18:38:56.615 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-10T18:38:56.613+0000 7fcec81fb640 1 -- 172.21.15.175:0/2434105502 wait complete. 2024-09-10T18:38:57.858 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:57.858 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.858 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.858 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.858 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:38:57.859 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.175:0/2434105502' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:57.859 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:57.859 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:38:57.859 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.859 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ignoring --setuser ceph since I am not root 2024-09-10T18:38:57.860 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ignoring --setgroup ceph since I am not root 2024-09-10T18:38:57.860 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:57.738+0000 7f472a832ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:38:57.860 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:57.791+0000 7f472a832ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:38:57.860 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:57.861 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.861 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.861 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.861 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:38:57.861 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.175:0/2434105502' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:57.862 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:57.862 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:38:57.862 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.888 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: pgmap v197: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:38:57.888 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.175:0/2434105502' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='client.24502 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='client.24502 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:38:57.889 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' 2024-09-10T18:38:57.890 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ignoring --setuser ceph since I am not root 2024-09-10T18:38:57.890 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ignoring --setgroup ceph since I am not root 2024-09-10T18:38:57.890 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:57.768+0000 7fa641becec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:38:57.890 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:57.822+0000 7fa641becec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:38:57.890 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:57 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:57.886+0000 7fa641becec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:38:58.119 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:57.855+0000 7f472a832ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:38:58.120 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:57.982+0000 7f472a832ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:38:58.151 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:58.013+0000 7fa641becec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:38:58.151 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:58.149+0000 7fa641becec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:38:58.385 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:58.117+0000 7f472a832ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:38:58.459 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-10T18:38:58.459 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:38:58.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:38:58.501 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:38:58.501 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:38:58.501 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-mon[33399]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-mon[33399]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:58.401+0000 7fa641becec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:58.466+0000 7fa641becec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:38:58.677 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:58 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:58.632+0000 7fa641becec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:38:58.771 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:38:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-mon[29369]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:38:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-mon[29369]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:38:58.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-mon[40885]: from='mgr.14150 v1:172.21.15.131:0/1973975300' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:38:58.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-mon[40885]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:38:58.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:58.388+0000 7f472a832ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:38:58.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:58.451+0000 7f472a832ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:38:58.828 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:58.628+0000 7f472a832ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:38:59.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:59 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:59.547+0000 7f472a832ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:38:59.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:38:59 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:38:59.615+0000 7f472a832ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:38:59.927 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:59 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:59.474+0000 7fa641becec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:38:59.927 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:38:59 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:38:59.539+0000 7fa641becec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:39:00.523 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.236+0000 7fa641becec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:39:00.523 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.344+0000 7fa641becec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:39:00.523 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.462+0000 7fa641becec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:39:00.559 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.341+0000 7f472a832ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:39:00.559 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.453+0000 7f472a832ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:39:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.557+0000 7f9d832ef640 1 -- 172.21.15.131:0/848760835 >> v1:172.21.15.131:6790/0 conn(0x7f9d7c101530 legacy=0x7f9d7c101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.557+0000 7f9d832ef640 1 -- 172.21.15.131:0/848760835 shutdown_connections 2024-09-10T18:39:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.557+0000 7f9d832ef640 1 -- 172.21.15.131:0/848760835 >> 172.21.15.131:0/848760835 conn(0x7f9d7c0752c0 msgr2=0x7f9d7c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.557+0000 7f9d832ef640 1 -- 172.21.15.131:0/848760835 shutdown_connections 2024-09-10T18:39:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.557+0000 7f9d832ef640 1 -- 172.21.15.131:0/848760835 wait complete. 2024-09-10T18:39:00.611 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d832ef640 1 Processor -- start 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d832ef640 1 -- start start 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d832ef640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9d7c11bf60 con 0x7f9d7c101530 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d832ef640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9d7c11d120 con 0x7f9d7c1042a0 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d832ef640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9d7c11e320 con 0x7f9d7c107e00 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d8108b640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9d7c101530 0x7f9d7c065010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34612/0 (socket says 172.21.15.131:34612) 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.558+0000 7f9d8108b640 1 -- 172.21.15.131:0/3571412414 learned_addr learned my addr 172.21.15.131:0/3571412414 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465753360 0 0) 0x7f9d7c11bf60 con 0x7f9d7c101530 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d680034e0 con 0x7f9d7c101530 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3728580555 0 0) 0x7f9d7c11e320 con 0x7f9d7c107e00 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7c11bf60 con 0x7f9d7c107e00 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 246463673 0 0) 0x7f9d7c11d120 con 0x7f9d7c1042a0 2024-09-10T18:39:00.612 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d7c11e320 con 0x7f9d7c1042a0 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1965747851 0 0) 0x7f9d680034e0 con 0x7f9d7c101530 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d7c11d120 con 0x7f9d7c101530 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9d78001440 con 0x7f9d7c101530 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3697175312 0 0) 0x7f9d7c11e320 con 0x7f9d7c1042a0 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d680034e0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9d6c0018c0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 846340094 0 0) 0x7f9d7c11bf60 con 0x7f9d7c107e00 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9d7c11e320 con 0x7f9d7c107e00 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9d74001bb0 con 0x7f9d7c107e00 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2453195232 0 0) 0x7f9d680034e0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 >> v1:172.21.15.131:6790/0 conn(0x7f9d7c107e00 legacy=0x7f9d7c11a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:00.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 >> v1:172.21.15.131:6789/0 conn(0x7f9d7c101530 legacy=0x7f9d7c065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7c11f520 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d7c11e510 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f9d6c0022a0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9d6c0030d0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.559+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d7c11e9e0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.560+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7c0fcc30 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.561+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 17) v1 ==== 90811+0+0 (unknown 1429559844 0 0) 0x7f9d6c0195f0 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.561+0000 7f9d8108b640 1 -- 172.21.15.131:0/3571412414 >> v1:172.21.15.131:6800/3123609852 conn(0x7f9d6806fd70 legacy=0x7f9d68072220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/3123609852 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.561+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(60..60 src has 1..60) v4 ==== 5938+0+0 (unknown 1429322001 0 0) 0x7f9d6c087620 con 0x7f9d7c1042a0 2024-09-10T18:39:00.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.565+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f9d6c054290 con 0x7f9d7c1042a0 2024-09-10T18:39:00.763 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.762+0000 7f9d8108b640 1 -- 172.21.15.131:0/3571412414 >> v1:172.21.15.131:6800/3123609852 conn(0x7f9d6806fd70 legacy=0x7f9d68072220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.131:6800/3123609852 2024-09-10T18:39:00.770 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.769+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f9d7c0ffd80 con 0x7f9d7c1042a0 2024-09-10T18:39:00.773 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.772+0000 7f9d71ffb640 1 -- 172.21.15.131:0/3571412414 <== mon.1 v1:172.21.15.175:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+176677 (unknown 831476310 0 4211989349) 0x7f9d6c053c30 con 0x7f9d7c1042a0 2024-09-10T18:39:00.774 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.521+0000 7fa641becec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:39:00.774 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.647+0000 7fa641becec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:39:00.774 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.713+0000 7fa641becec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:39:00.775 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.772+0000 7fa641becec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:39:00.777 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:39:00.778 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.775+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 >> v1:172.21.15.131:6800/3123609852 conn(0x7f9d6806fd70 legacy=0x7f9d68072220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:39:00.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.775+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 >> v1:172.21.15.175:6789/0 conn(0x7f9d7c1042a0 legacy=0x7f9d7c065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:00.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.775+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 shutdown_connections 2024-09-10T18:39:00.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.775+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 >> 172.21.15.131:0/3571412414 conn(0x7f9d7c0752c0 msgr2=0x7f9d7c0f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:00.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.775+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 shutdown_connections 2024-09-10T18:39:00.779 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:00.775+0000 7f9d832ef640 1 -- 172.21.15.131:0/3571412414 wait complete. 2024-09-10T18:39:00.825 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.582+0000 7f472a832ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:39:00.826 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.644+0000 7f472a832ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:39:00.826 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.774+0000 7f472a832ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:39:01.035 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3571412414' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:39:01.035 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.907+0000 7fa641becec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:39:01.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:00 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:00.970+0000 7fa641becec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:39:01.036 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:01.033+0000 7fa641becec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:39:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3571412414' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:39:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3571412414' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:39:01.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.847+0000 7f472a832ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:39:01.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:00.905+0000 7f472a832ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:39:01.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:01.042+0000 7f472a832ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:39:01.371 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:01.107+0000 7f472a832ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:39:01.371 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:01.173+0000 7f472a832ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:39:01.371 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:01.254+0000 7f472a832ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:39:01.372 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:01.311+0000 7f472a832ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:39:01.372 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:01 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:01.369+0000 7f472a832ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:39:01.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:01.113+0000 7fa641becec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:39:01.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:01.170+0000 7fa641becec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:39:01.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:01.226+0000 7fa641becec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:39:01.551 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6800","nonce":3123609852}]},"active_addr":"172.21.15.131:6800/3123609852","active_change":"2024-09-10T18:34:02.347198+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.131:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":2180975428}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":758123301}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3330964657}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":4120844921}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.131:0","nonce":3867322050}]}]} 2024-09-10T18:39:01.555 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:39:01.555 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:39:01.556 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:39:01.802 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:02.212 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:02.162+0000 7f472a832ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[29369]: Standby manager daemon x restarted 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[29369]: Standby manager daemon x started 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[29369]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:39:02.213 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[40885]: Standby manager daemon x restarted 2024-09-10T18:39:02.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[40885]: Standby manager daemon x started 2024-09-10T18:39:02.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:39:02.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:39:02.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:39:02.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-mon[40885]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:39:02.372 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-mon[33399]: Standby manager daemon x restarted 2024-09-10T18:39:02.372 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-mon[33399]: Standby manager daemon x started 2024-09-10T18:39:02.372 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-mon[33399]: from='mgr.? v1:172.21.15.175:0/668206927' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:01.965+0000 7fa641becec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: [10/Sep/2024:18:39:01] ENGINE Bus STARTING 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: CherryPy Checker: 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: The Application mounted at '' has an empty config. 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:01 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: [10/Sep/2024:18:39:02] ENGINE Serving on http://:::9283 2024-09-10T18:39:02.373 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:02 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: [10/Sep/2024:18:39:02] ENGINE Bus STARTED 2024-09-10T18:39:02.508 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:02] ENGINE Bus STARTING 2024-09-10T18:39:02.508 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: CherryPy Checker: 2024-09-10T18:39:02.508 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: The Application mounted at '' has an empty config. 2024-09-10T18:39:02.508 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:02.508 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:02] ENGINE Serving on http://:::9283 2024-09-10T18:39:02.508 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:02 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:02] ENGINE Bus STARTED 2024-09-10T18:39:03.307 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: Active manager daemon y restarted 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: Activating manager daemon y 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: mgrmap e19: y(active, starting, since 0.011074s), standbys: x 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:39:03.308 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: Manager daemon y is now available 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:03.309 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: Active manager daemon y restarted 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: Activating manager daemon y 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: mgrmap e19: y(active, starting, since 0.011074s), standbys: x 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:39:03.310 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:39:03.311 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: Manager daemon y is now available 2024-09-10T18:39:03.312 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:03.312 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:03.312 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:03.312 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:39:03.312 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:39:03.345 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: Active manager daemon y restarted 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: Activating manager daemon y 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: mgrmap e19: y(active, starting, since 0.011074s), standbys: x 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:39:03.346 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:39:03.347 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:39:03.348 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: Manager daemon y is now available 2024-09-10T18:39:03.348 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:03.348 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:03.348 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:03.348 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:39:03.348 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:39:03.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.522+0000 7f7317a76640 1 -- 172.21.15.131:0/3578693207 >> v1:172.21.15.131:6790/0 conn(0x7f73101042a0 legacy=0x7f7310106680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:03.524 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.522+0000 7f7317a76640 1 -- 172.21.15.131:0/3578693207 shutdown_connections 2024-09-10T18:39:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.522+0000 7f7317a76640 1 -- 172.21.15.131:0/3578693207 >> 172.21.15.131:0/3578693207 conn(0x7f73100752c0 msgr2=0x7f73100f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.522+0000 7f7317a76640 1 -- 172.21.15.131:0/3578693207 shutdown_connections 2024-09-10T18:39:03.525 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7317a76640 1 -- 172.21.15.131:0/3578693207 wait complete. 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7317a76640 1 Processor -- start 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7317a76640 1 -- start start 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7317a76640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f731011bea0 con 0x7f7310107e00 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7317a76640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f731011d0a0 con 0x7f7310101530 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7317a76640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f731011e2a0 con 0x7f73101042a0 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7316013640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7310107e00 0x7f731011a5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:34712/0 (socket says 172.21.15.131:34712) 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7316013640 1 -- 172.21.15.131:0/812461593 learned_addr learned my addr 172.21.15.131:0/812461593 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.523+0000 7f7315011640 1 --1- 172.21.15.131:0/812461593 >> v1:172.21.15.131:6790/0 conn(0x7f73101042a0 0x7f7310065720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:41998/0 (socket says 172.21.15.131:41998) 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843914114 0 0) 0x7f731011bea0 con 0x7f7310107e00 2024-09-10T18:39:03.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72fc0034e0 con 0x7f7310107e00 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818786246 0 0) 0x7f731011e2a0 con 0x7f73101042a0 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f731011bea0 con 0x7f73101042a0 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079518812 0 0) 0x7f731011d0a0 con 0x7f7310101530 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f731011e2a0 con 0x7f7310101530 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1277753652 0 0) 0x7f72fc0034e0 con 0x7f7310107e00 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f731011d0a0 con 0x7f7310107e00 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f7308002c50 con 0x7f7310107e00 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1004662690 0 0) 0x7f731011bea0 con 0x7f73101042a0 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f72fc0034e0 con 0x7f73101042a0 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 573871 0 0) 0x7f731011e2a0 con 0x7f7310101530 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f731011bea0 con 0x7f7310101530 2024-09-10T18:39:03.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f7300001430 con 0x7f73101042a0 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f730c001830 con 0x7f7310101530 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2722325737 0 0) 0x7f731011d0a0 con 0x7f7310107e00 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 >> v1:172.21.15.131:6790/0 conn(0x7f73101042a0 legacy=0x7f7310065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 >> v1:172.21.15.175:6789/0 conn(0x7f7310101530 legacy=0x7f7310065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731011f4a0 con 0x7f7310107e00 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731011e490 con 0x7f7310107e00 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f731011e960 con 0x7f7310107e00 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f7308001b40 con 0x7f7310107e00 2024-09-10T18:39:03.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.524+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f7308005d20 con 0x7f7310107e00 2024-09-10T18:39:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.525+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f4005010 con 0x7f7310107e00 2024-09-10T18:39:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.526+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 20) v1 ==== 90554+0+0 (unknown 2283736816 0 0) 0x7f73080016f0 con 0x7f7310107e00 2024-09-10T18:39:03.530 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.526+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f7308089ba0 con 0x7f7310107e00 2024-09-10T18:39:03.532 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.531+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f7308052ce0 con 0x7f7310107e00 2024-09-10T18:39:03.703 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.700+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72f40052c0 con 0x7f7310107e00 2024-09-10T18:39:03.704 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.701+0000 7f73067fc640 1 -- 172.21.15.131:0/812461593 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+19276 (unknown 1877459060 0 2300971104) 0x7f7308056580 con 0x7f7310107e00 2024-09-10T18:39:03.705 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:39:03.706 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":61,"fsid":"0846fc60-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:33:07.857701+0000","modified":"2024-09-10T18:39:02.165507+0000","last_up_change":"2024-09-10T18:38:12.634608+0000","last_in_change":"2024-09-10T18:37:56.112167+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:36:16.640762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-10T18:38:24.370563+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-10T18:38:24.606232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:38:26.208467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:38:28.176203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:38:30.243369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6801","nonce":437930310}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6802","nonce":437930310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6804","nonce":437930310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6803","nonce":437930310}]},"public_addr":"172.21.15.131:6801/437930310","cluster_addr":"172.21.15.131:6802/437930310","heartbeat_back_addr":"172.21.15.131:6804/437930310","heartbeat_front_addr":"172.21.15.131:6803/437930310","state":["exists","up"]},{"osd":1,"uuid":"20feda32-c9c6-42f4-8226-1652eccf014e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6805","nonce":2444922616}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6806","nonce":2444922616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6808","nonce":2444922616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6807","nonce":2444922616}]},"public_addr":"172.21.15.131:6805/2444922616","cluster_addr":"172.21.15.131:6806/2444922616","heartbeat_back_addr":"172.21.15.131:6808/2444922616","heartbeat_front_addr":"172.21.15.131:6807/2444922616","state":["exists","up"]},{"osd":2,"uuid":"bd6bffb1-379f-41bc-9600-f7aed3a9c7d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6809","nonce":1436967876}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6810","nonce":1436967876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6812","nonce":1436967876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6811","nonce":1436967876}]},"public_addr":"172.21.15.131:6809/1436967876","cluster_addr":"172.21.15.131:6810/1436967876","heartbeat_back_addr":"172.21.15.131:6812/1436967876","heartbeat_front_addr":"172.21.15.131:6811/1436967876","state":["exists","up"]},{"osd":3,"uuid":"5b888d2e-d94c-4cd8-afad-498c49a22a4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6813","nonce":2850347725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6814","nonce":2850347725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6816","nonce":2850347725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6815","nonce":2850347725}]},"public_addr":"172.21.15.131:6813/2850347725","cluster_addr":"172.21.15.131:6814/2850347725","heartbeat_back_addr":"172.21.15.131:6816/2850347725","heartbeat_front_addr":"172.21.15.131:6815/2850347725","state":["exists","up"]},{"osd":4,"uuid":"638c463c-43c6-4ca5-b221-de408d307972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6800","nonce":2653212041}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6801","nonce":2653212041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6803","nonce":2653212041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6802","nonce":2653212041}]},"public_addr":"172.21.15.175:6800/2653212041","cluster_addr":"172.21.15.175:6801/2653212041","heartbeat_back_addr":"172.21.15.175:6803/2653212041","heartbeat_front_addr":"172.21.15.175:6802/2653212041","state":["exists","up"]},{"osd":5,"uuid":"c07e1e1e-2ad2-4acb-b8d2-0a77b6661604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6804","nonce":210062365}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6805","nonce":210062365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6807","nonce":210062365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6806","nonce":210062365}]},"public_addr":"172.21.15.175:6804/210062365","cluster_addr":"172.21.15.175:6805/210062365","heartbeat_back_addr":"172.21.15.175:6807/210062365","heartbeat_front_addr":"172.21.15.175:6806/210062365","state":["exists","up"]},{"osd":6,"uuid":"8d8b5fe9-ef7d-4b85-a61b-343a3e69a751","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6808","nonce":3246498802}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6809","nonce":3246498802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6811","nonce":3246498802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6810","nonce":3246498802}]},"public_addr":"172.21.15.175:6808/3246498802","cluster_addr":"172.21.15.175:6809/3246498802","heartbeat_back_addr":"172.21.15.175:6811/3246498802","heartbeat_front_addr":"172.21.15.175:6810/3246498802","state":["exists","up"]},{"osd":7,"uuid":"bbba104c-8b89-4955-8189-3aa160932f1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6812","nonce":2356238523}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6813","nonce":2356238523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6815","nonce":2356238523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6814","nonce":2356238523}]},"public_addr":"172.21.15.175:6812/2356238523","cluster_addr":"172.21.15.175:6813/2356238523","heartbeat_back_addr":"172.21.15.175:6815/2356238523","heartbeat_front_addr":"172.21.15.175:6814/2356238523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:35:25.814053+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:35:50.012887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:13.994960+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:37.876356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:59.595337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:23.108539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:47.058002+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:11.527856+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6800/3123609852":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/3002086193":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/3135886569":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/4120844921":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/968404841":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/758123301":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/681874383":"2024-09-11T18:33:39.624176+0000","172.21.15.131:6800/1625784054":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/3549403922":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/1393657916":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/3330964657":"2024-09-11T18:39:02.165470+0000","172.21.15.131:6800/2393239631":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/2180975428":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/3867322050":"2024-09-11T18:39:02.165470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:39:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.703+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 >> v1:172.21.15.131:6800/336444945 conn(0x7f72fc06fc60 legacy=0x7f72fc072110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:03.706 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.703+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 >> v1:172.21.15.131:6789/0 conn(0x7f7310107e00 legacy=0x7f731011a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:03.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.704+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 shutdown_connections 2024-09-10T18:39:03.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.704+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 >> 172.21.15.131:0/812461593 conn(0x7f73100752c0 msgr2=0x7f731010b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:03.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.704+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 shutdown_connections 2024-09-10T18:39:03.707 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:03.704+0000 7f7317a76640 1 -- 172.21.15.131:0/812461593 wait complete. 2024-09-10T18:39:04.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[29369]: mgrmap e20: y(active, since 1.01429s), standbys: x 2024-09-10T18:39:04.267 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[29369]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/812461593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[40885]: mgrmap e20: y(active, since 1.01429s), standbys: x 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[40885]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:04.268 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/812461593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:04.053+0000 7f46cd5f4640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:39:04] ENGINE Error in HTTPServer.serve 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: Traceback (most recent call last): 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._connections.run(self.expiration_interval) 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._run(expiration_interval) 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:39:04.269 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: s = self.context.wrap_socket( 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: return self.sslsocket_class._create( 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self.do_handshake() 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: self._sslobj.do_handshake() 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:39:04.270 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:04 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:04 smithi175 ceph-mon[33399]: mgrmap e20: y(active, since 1.01429s), standbys: x 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:04 smithi175 ceph-mon[33399]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/812461593' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:04 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:05.131 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:39:05.131 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:39:05.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:39:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:05.405 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:05.506 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[29369]: [10/Sep/2024:18:39:03] ENGINE Bus STARTING 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[29369]: [10/Sep/2024:18:39:04] ENGINE Serving on https://172.21.15.131:7150 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[29369]: [10/Sep/2024:18:39:04] ENGINE Bus STARTED 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[29369]: [10/Sep/2024:18:39:04] ENGINE Error in HTTPServer.serve 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: Traceback (most recent call last): 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._run(expiration_interval) 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: s = self.context.wrap_socket( 2024-09-10T18:39:05.507 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout: return self.sslsocket_class._create( 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout: self.do_handshake() 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout: self._sslobj.do_handshake() 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[29369]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[29369]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[40885]: [10/Sep/2024:18:39:03] ENGINE Bus STARTING 2024-09-10T18:39:05.508 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[40885]: [10/Sep/2024:18:39:04] ENGINE Serving on https://172.21.15.131:7150 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[40885]: [10/Sep/2024:18:39:04] ENGINE Bus STARTED 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[40885]: [10/Sep/2024:18:39:04] ENGINE Error in HTTPServer.serve 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: Traceback (most recent call last): 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._run(expiration_interval) 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:39:05.509 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: s = self.context.wrap_socket( 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: return self.sslsocket_class._create( 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: self.do_handshake() 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: self._sslobj.do_handshake() 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[40885]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:05.510 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:05 smithi131 ceph-mon[40885]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-mon[33399]: [10/Sep/2024:18:39:03] ENGINE Bus STARTING 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-mon[33399]: [10/Sep/2024:18:39:04] ENGINE Serving on https://172.21.15.131:7150 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-mon[33399]: [10/Sep/2024:18:39:04] ENGINE Bus STARTED 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-mon[33399]: [10/Sep/2024:18:39:04] ENGINE Error in HTTPServer.serve 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout: Traceback (most recent call last): 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:39:05.677 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._run(expiration_interval) 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: s = self.context.wrap_socket( 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: return self.sslsocket_class._create( 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: self.do_handshake() 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:39:05.678 INFO:journalctl@ceph.mon.b.smithi175.stdout: self._sslobj.do_handshake() 2024-09-10T18:39:05.679 INFO:journalctl@ceph.mon.b.smithi175.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:39:05.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-mon[33399]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:05.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:05 smithi175 ceph-mon[33399]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:06.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:06 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:06 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:07.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-10T18:39:07.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:07.360 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi131 to 3760M 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.361 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi131 to 3760M 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.362 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:07 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.358+0000 7f0f8fd34640 1 -- 172.21.15.131:0/834747818 >> v1:172.21.15.131:6790/0 conn(0x7f0f881042a0 legacy=0x7f0f88106680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:07.395 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.358+0000 7f0f8fd34640 1 -- 172.21.15.131:0/834747818 shutdown_connections 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.358+0000 7f0f8fd34640 1 -- 172.21.15.131:0/834747818 >> 172.21.15.131:0/834747818 conn(0x7f0f880752c0 msgr2=0x7f0f880f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.358+0000 7f0f8fd34640 1 -- 172.21.15.131:0/834747818 shutdown_connections 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.358+0000 7f0f8fd34640 1 -- 172.21.15.131:0/834747818 wait complete. 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8fd34640 1 Processor -- start 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8fd34640 1 -- start start 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8fd34640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f8806f4e0 con 0x7f0f88101530 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8fd34640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f8806f6d0 con 0x7f0f88107e00 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8fd34640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f88126b50 con 0x7f0f881042a0 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8e2d1640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0f88107e00 0x7f0f88123410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.131:45020/0 (socket says 172.21.15.131:45020) 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f8e2d1640 1 -- 172.21.15.131:0/3564462966 learned_addr learned my addr 172.21.15.131:0/3564462966 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.359+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537815126 0 0) 0x7f0f8806f6d0 con 0x7f0f88107e00 2024-09-10T18:39:07.396 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f74003120 con 0x7f0f88107e00 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043605268 0 0) 0x7f0f8806f4e0 con 0x7f0f88101530 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f8806f6d0 con 0x7f0f88101530 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173381398 0 0) 0x7f0f88126b50 con 0x7f0f881042a0 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f8806f4e0 con 0x7f0f881042a0 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2167161907 0 0) 0x7f0f74003120 con 0x7f0f88107e00 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f88126b50 con 0x7f0f88107e00 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0f80002cc0 con 0x7f0f88107e00 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3025258037 0 0) 0x7f0f8806f6d0 con 0x7f0f88101530 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f74003120 con 0x7f0f88101530 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0f840019f0 con 0x7f0f88101530 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 768965141 0 0) 0x7f0f8806f4e0 con 0x7f0f881042a0 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f8806f6d0 con 0x7f0f881042a0 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0f78002c20 con 0x7f0f881042a0 2024-09-10T18:39:07.397 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1742038841 0 0) 0x7f0f88126b50 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 >> v1:172.21.15.131:6790/0 conn(0x7f0f881042a0 legacy=0x7f0f88065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 >> v1:172.21.15.131:6789/0 conn(0x7f0f88101530 legacy=0x7f0f88065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88127cf0 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f88126d40 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f88127190 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f0f80002ee0 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.360+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f0f80005d90 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.361+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f880fcc30 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.361+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f0f80006040 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.362+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f0f8008a320 con 0x7f0f88107e00 2024-09-10T18:39:07.398 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.366+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f0f80057010 con 0x7f0f88107e00 2024-09-10T18:39:07.541 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.539+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f880ffcc0 con 0x7f0f88107e00 2024-09-10T18:39:07.542 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.540+0000 7f0f7e7fc640 1 -- 172.21.15.131:0/3564462966 <== mon.1 v1:172.21.15.175:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+19276 (unknown 1877459060 0 2300971104) 0x7f0f800569b0 con 0x7f0f88107e00 2024-09-10T18:39:07.542 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:39:07.543 INFO:teuthology.orchestra.run.smithi131.stdout:{"epoch":61,"fsid":"0846fc60-6fa3-11ef-bcea-c7b262605968","created":"2024-09-10T18:33:07.857701+0000","modified":"2024-09-10T18:39:02.165507+0000","last_up_change":"2024-09-10T18:38:12.634608+0000","last_in_change":"2024-09-10T18:37:56.112167+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:36:16.640762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-10T18:38:24.370563+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-10T18:38:24.606232+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:38:26.208467+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:38:28.176203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:38:30.243369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"bd23f81e-cd1c-4ac5-8fa6-a0999b8d3c01","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6801","nonce":437930310}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6802","nonce":437930310}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6804","nonce":437930310}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6803","nonce":437930310}]},"public_addr":"172.21.15.131:6801/437930310","cluster_addr":"172.21.15.131:6802/437930310","heartbeat_back_addr":"172.21.15.131:6804/437930310","heartbeat_front_addr":"172.21.15.131:6803/437930310","state":["exists","up"]},{"osd":1,"uuid":"20feda32-c9c6-42f4-8226-1652eccf014e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6805","nonce":2444922616}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6806","nonce":2444922616}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6808","nonce":2444922616}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6807","nonce":2444922616}]},"public_addr":"172.21.15.131:6805/2444922616","cluster_addr":"172.21.15.131:6806/2444922616","heartbeat_back_addr":"172.21.15.131:6808/2444922616","heartbeat_front_addr":"172.21.15.131:6807/2444922616","state":["exists","up"]},{"osd":2,"uuid":"bd6bffb1-379f-41bc-9600-f7aed3a9c7d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6809","nonce":1436967876}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6810","nonce":1436967876}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6812","nonce":1436967876}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6811","nonce":1436967876}]},"public_addr":"172.21.15.131:6809/1436967876","cluster_addr":"172.21.15.131:6810/1436967876","heartbeat_back_addr":"172.21.15.131:6812/1436967876","heartbeat_front_addr":"172.21.15.131:6811/1436967876","state":["exists","up"]},{"osd":3,"uuid":"5b888d2e-d94c-4cd8-afad-498c49a22a4b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6813","nonce":2850347725}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6814","nonce":2850347725}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6816","nonce":2850347725}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.131:6815","nonce":2850347725}]},"public_addr":"172.21.15.131:6813/2850347725","cluster_addr":"172.21.15.131:6814/2850347725","heartbeat_back_addr":"172.21.15.131:6816/2850347725","heartbeat_front_addr":"172.21.15.131:6815/2850347725","state":["exists","up"]},{"osd":4,"uuid":"638c463c-43c6-4ca5-b221-de408d307972","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6800","nonce":2653212041}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6801","nonce":2653212041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6803","nonce":2653212041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6802","nonce":2653212041}]},"public_addr":"172.21.15.175:6800/2653212041","cluster_addr":"172.21.15.175:6801/2653212041","heartbeat_back_addr":"172.21.15.175:6803/2653212041","heartbeat_front_addr":"172.21.15.175:6802/2653212041","state":["exists","up"]},{"osd":5,"uuid":"c07e1e1e-2ad2-4acb-b8d2-0a77b6661604","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6804","nonce":210062365}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6805","nonce":210062365}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6807","nonce":210062365}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6806","nonce":210062365}]},"public_addr":"172.21.15.175:6804/210062365","cluster_addr":"172.21.15.175:6805/210062365","heartbeat_back_addr":"172.21.15.175:6807/210062365","heartbeat_front_addr":"172.21.15.175:6806/210062365","state":["exists","up"]},{"osd":6,"uuid":"8d8b5fe9-ef7d-4b85-a61b-343a3e69a751","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6808","nonce":3246498802}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6809","nonce":3246498802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6811","nonce":3246498802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6810","nonce":3246498802}]},"public_addr":"172.21.15.175:6808/3246498802","cluster_addr":"172.21.15.175:6809/3246498802","heartbeat_back_addr":"172.21.15.175:6811/3246498802","heartbeat_front_addr":"172.21.15.175:6810/3246498802","state":["exists","up"]},{"osd":7,"uuid":"bbba104c-8b89-4955-8189-3aa160932f1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6812","nonce":2356238523}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6813","nonce":2356238523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6815","nonce":2356238523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.175:6814","nonce":2356238523}]},"public_addr":"172.21.15.175:6812/2356238523","cluster_addr":"172.21.15.175:6813/2356238523","heartbeat_back_addr":"172.21.15.175:6815/2356238523","heartbeat_front_addr":"172.21.15.175:6814/2356238523","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:35:25.814053+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:35:50.012887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:13.994960+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:37.876356+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:59.595337+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:23.108539+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:47.058002+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:11.527856+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.131:6800/3123609852":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/3002086193":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/3135886569":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/4120844921":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/968404841":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/758123301":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/681874383":"2024-09-11T18:33:39.624176+0000","172.21.15.131:6800/1625784054":"2024-09-11T18:33:39.624176+0000","172.21.15.131:0/3549403922":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/1393657916":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/3330964657":"2024-09-11T18:39:02.165470+0000","172.21.15.131:6800/2393239631":"2024-09-11T18:34:02.347093+0000","172.21.15.131:0/2180975428":"2024-09-11T18:39:02.165470+0000","172.21.15.131:0/3867322050":"2024-09-11T18:39:02.165470+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:39:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.543+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 >> v1:172.21.15.131:6800/336444945 conn(0x7f0f7406fe70 legacy=0x7f0f74072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.543+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 >> v1:172.21.15.175:6789/0 conn(0x7f0f88107e00 legacy=0x7f0f88123410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.543+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 shutdown_connections 2024-09-10T18:39:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.543+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 >> 172.21.15.131:0/3564462966 conn(0x7f0f880752c0 msgr2=0x7f0f8810b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.543+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 shutdown_connections 2024-09-10T18:39:07.545 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:07.543+0000 7f0f8fd34640 1 -- 172.21.15.131:0/3564462966 wait complete. 2024-09-10T18:39:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi175 to 3760M 2024-09-10T18:39:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi131 to 3760M 2024-09-10T18:39:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: Updating smithi131:/etc/ceph/ceph.conf 2024-09-10T18:39:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: Updating smithi175:/etc/ceph/ceph.conf 2024-09-10T18:39:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:07.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:07 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:08.100 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:39:08.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:39:08.101 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:39:08.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:39:08.102 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:39:08.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:39:08.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:39:08.103 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:39:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[29369]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[29369]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[29369]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[29369]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[29369]: Deploying daemon alertmanager.a on smithi131 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3564462966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[40885]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[40885]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[40885]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[40885]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[40885]: Deploying daemon alertmanager.a on smithi131 2024-09-10T18:39:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3564462966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:08.651 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.652 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.653 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.654 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.656 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.660 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.660 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.665 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:08 smithi175 ceph-mon[33399]: Updating smithi131:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:08 smithi175 ceph-mon[33399]: Updating smithi175:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:39:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:08 smithi175 ceph-mon[33399]: Updating smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:39:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:08 smithi175 ceph-mon[33399]: Updating smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:39:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:08 smithi175 ceph-mon[33399]: Deploying daemon alertmanager.a on smithi131 2024-09-10T18:39:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3564462966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:09 smithi131 ceph-mon[29369]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T18:39:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:09 smithi131 ceph-mon[40885]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T18:39:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:09 smithi175 ceph-mon[33399]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T18:39:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:11 smithi131 ceph-mon[29369]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T18:39:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:11 smithi131 ceph-mon[40885]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T18:39:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:11 smithi175 ceph-mon[33399]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T18:39:13.382 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:13 smithi131 ceph-mon[29369]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:39:13.383 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:13 smithi131 ceph-mon[40885]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:39:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:13 smithi175 ceph-mon[33399]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:39:13.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:39:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:39:14.494 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:14.494 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:15 smithi175 ceph-mon[33399]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:39:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:39:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:15 smithi131 ceph-mon[40885]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:39:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:15 smithi131 ceph-mon[29369]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:39:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:17 smithi131 ceph-mon[29369]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:39:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:17 smithi131 ceph-mon[40885]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:39:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:17 smithi175 ceph-mon[33399]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:39:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:18.826 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.793+0000 7fb7a8796640 1 -- 172.21.15.131:0/3311286138 >> v1:172.21.15.131:6790/0 conn(0x7fb7a0105310 legacy=0x7fb7a01076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.794+0000 7fb7a8796640 1 -- 172.21.15.131:0/3311286138 shutdown_connections 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.794+0000 7fb7a8796640 1 -- 172.21.15.131:0/3311286138 >> 172.21.15.131:0/3311286138 conn(0x7fb7a00f8110 msgr2=0x7fb7a00fa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.794+0000 7fb7a8796640 1 -- 172.21.15.131:0/3311286138 shutdown_connections 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.794+0000 7fb7a8796640 1 -- 172.21.15.131:0/3311286138 wait complete. 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.794+0000 7fb7a8796640 1 Processor -- start 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.794+0000 7fb7a8796640 1 -- start start 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb7a8796640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7a011bf10 con 0x7fb7a0108ea0 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb7a8796640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7a011d0d0 con 0x7fb7a0102550 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb7a8796640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7a011e2d0 con 0x7fb7a0105310 2024-09-10T18:39:18.827 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb7a5d31640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fb7a0105310 0x7fb7a00fecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:58730/0 (socket says 172.21.15.131:58730) 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb7a5d31640 1 -- 172.21.15.131:0/4067763540 learned_addr learned my addr 172.21.15.131:0/4067763540 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447720486 0 0) 0x7fb7a011bf10 con 0x7fb7a0108ea0 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb78c0034e0 con 0x7fb7a0108ea0 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 770340174 0 0) 0x7fb7a011e2d0 con 0x7fb7a0105310 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a011bf10 con 0x7fb7a0105310 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783668940 0 0) 0x7fb7a011d0d0 con 0x7fb7a0102550 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7a011e2d0 con 0x7fb7a0102550 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1617637874 0 0) 0x7fb7a011bf10 con 0x7fb7a0105310 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7a011d0d0 con 0x7fb7a0105310 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb7900010b0 con 0x7fb7a0105310 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1907629969 0 0) 0x7fb78c0034e0 con 0x7fb7a0108ea0 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7a011bf10 con 0x7fb7a0108ea0 2024-09-10T18:39:18.828 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb798001ca0 con 0x7fb7a0108ea0 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2804063040 0 0) 0x7fb7a011e2d0 con 0x7fb7a0102550 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb78c0034e0 con 0x7fb7a0102550 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb79c001840 con 0x7fb7a0102550 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1930896995 0 0) 0x7fb7a011d0d0 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 >> v1:172.21.15.175:6789/0 conn(0x7fb7a0102550 legacy=0x7fb7a00fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 >> v1:172.21.15.131:6789/0 conn(0x7fb7a0108ea0 legacy=0x7fb7a011a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.795+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a011f4d0 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.796+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a011e4c0 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.796+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a011e910 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.796+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fb790001580 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.796+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb790003540 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.796+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb784000f20 con 0x7fb7a0105310 2024-09-10T18:39:18.829 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.797+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fb7900036a0 con 0x7fb7a0105310 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.798+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fb79008a130 con 0x7fb7a0105310 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.798+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6813/2850347725 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb78c07e2d0 con 0x7fb78c07a630 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.798+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7fb79008a560 con 0x7fb7a0105310 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.799+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== osd.3 v1:172.21.15.131:6813/2850347725 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fb78c07e2d0 con 0x7fb78c07a630 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.811+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 --> v1:172.21.15.131:6813/2850347725 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb784002b10 con 0x7fb78c07a630 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.811+0000 7fb796ffd640 1 -- 172.21.15.131:0/4067763540 <== osd.3 v1:172.21.15.131:6813/2850347725 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3350110986) 0x7fb784002b10 con 0x7fb78c07a630 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 >> v1:172.21.15.131:6813/2850347725 conn(0x7fb78c07a630 legacy=0x7fb78c07ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:18.830 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 >> v1:172.21.15.131:6800/336444945 conn(0x7fb78c06fe30 legacy=0x7fb78c0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 >> v1:172.21.15.131:6790/0 conn(0x7fb7a0105310 legacy=0x7fb7a00fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a6d33640 1 -- 172.21.15.131:0/4067763540 reap_dead start 2024-09-10T18:39:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 shutdown_connections 2024-09-10T18:39:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 >> 172.21.15.131:0/4067763540 conn(0x7fb7a00f8110 msgr2=0x7fb7a0064c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 shutdown_connections 2024-09-10T18:39:18.831 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.812+0000 7fb7a8796640 1 -- 172.21.15.131:0/4067763540 wait complete. 2024-09-10T18:39:19.508 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:19 smithi131 ceph-mon[29369]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:39:19.508 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:19 smithi131 ceph-mon[40885]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.505+0000 7fe77c174640 1 -- 172.21.15.131:0/3454150294 >> v1:172.21.15.131:6789/0 conn(0x7fe774108ed0 legacy=0x7fe77410b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.505+0000 7fe77c174640 1 -- 172.21.15.131:0/3454150294 shutdown_connections 2024-09-10T18:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.505+0000 7fe77c174640 1 -- 172.21.15.131:0/3454150294 >> 172.21.15.131:0/3454150294 conn(0x7fe7740f8140 msgr2=0x7fe7740fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 -- 172.21.15.131:0/3454150294 shutdown_connections 2024-09-10T18:39:19.613 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 -- 172.21.15.131:0/3454150294 wait complete. 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 Processor -- start 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 -- start start 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe77411bfd0 con 0x7fe774105340 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe77411d190 con 0x7fe774108ed0 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe77c174640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe77411e390 con 0x7fe774102580 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.506+0000 7fe779f10640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fe774102580 0x7fe7740fe5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:41164/0 (socket says 172.21.15.131:41164) 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe779f10640 1 -- 172.21.15.131:0/1168679265 learned_addr learned my addr 172.21.15.131:0/1168679265 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe77970f640 1 --1- 172.21.15.131:0/1168679265 >> v1:172.21.15.131:6789/0 conn(0x7fe774105340 0x7fe7740fecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59658/0 (socket says 172.21.15.131:59658) 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962733839 0 0) 0x7fe77411bfd0 con 0x7fe774105340 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7600034e0 con 0x7fe774105340 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2891195756 0 0) 0x7fe77411e390 con 0x7fe774102580 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe77411bfd0 con 0x7fe774102580 2024-09-10T18:39:19.614 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232859157 0 0) 0x7fe77411d190 con 0x7fe774108ed0 2024-09-10T18:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe77411e390 con 0x7fe774108ed0 2024-09-10T18:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1930523779 0 0) 0x7fe7600034e0 con 0x7fe774105340 2024-09-10T18:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe77411d190 con 0x7fe774105340 2024-09-10T18:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe7640019f0 con 0x7fe774105340 2024-09-10T18:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1121332744 0 0) 0x7fe77411bfd0 con 0x7fe774102580 2024-09-10T18:39:19.615 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe7600034e0 con 0x7fe774102580 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe770001880 con 0x7fe774102580 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1712801342 0 0) 0x7fe77411e390 con 0x7fe774108ed0 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe77411bfd0 con 0x7fe774108ed0 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe76c001420 con 0x7fe774108ed0 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 226705550 0 0) 0x7fe77411d190 con 0x7fe774105340 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 >> v1:172.21.15.131:6790/0 conn(0x7fe774102580 legacy=0x7fe7740fe5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 >> v1:172.21.15.175:6789/0 conn(0x7fe774108ed0 legacy=0x7fe77411a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe77411f590 con 0x7fe774105340 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77411c1c0 con 0x7fe774105340 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77411c610 con 0x7fe774105340 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fe7640023d0 con 0x7fe774105340 2024-09-10T18:39:19.616 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.507+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe764005900 con 0x7fe774105340 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.508+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe758000f20 con 0x7fe774105340 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.509+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fe764001f80 con 0x7fe774105340 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.510+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fe764089ce0 con 0x7fe774105340 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.510+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6801/437930310 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe76007e470 con 0x7fe76007a7d0 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.510+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7fe76408a080 con 0x7fe774105340 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.511+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== osd.0 v1:172.21.15.131:6801/437930310 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fe76007e470 con 0x7fe76007a7d0 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.525+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 --> v1:172.21.15.131:6801/437930310 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe758002b10 con 0x7fe76007a7d0 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe76a7fc640 1 -- 172.21.15.131:0/1168679265 <== osd.0 v1:172.21.15.131:6801/437930310 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2546019811) 0x7fe758002b10 con 0x7fe76007a7d0 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 >> v1:172.21.15.131:6801/437930310 conn(0x7fe76007a7d0 legacy=0x7fe76007cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 >> v1:172.21.15.131:6800/336444945 conn(0x7fe76006ff00 legacy=0x7fe7600723b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 >> v1:172.21.15.131:6789/0 conn(0x7fe774105340 legacy=0x7fe7740fecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77a711640 1 -- 172.21.15.131:0/1168679265 reap_dead start 2024-09-10T18:39:19.617 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 shutdown_connections 2024-09-10T18:39:19.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 >> 172.21.15.131:0/1168679265 conn(0x7fe7740f8140 msgr2=0x7fe7740f8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:19.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 shutdown_connections 2024-09-10T18:39:19.618 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.526+0000 7fe77c174640 1 -- 172.21.15.131:0/1168679265 wait complete. 2024-09-10T18:39:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:19 smithi175 ceph-mon[33399]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:39:20.307 INFO:teuthology.orchestra.run.smithi131.stdout:34359738416 2024-09-10T18:39:20.307 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:39:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:21 smithi131 ceph-mon[29369]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:21 smithi131 ceph-mon[40885]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:21 smithi175 ceph-mon[33399]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:21.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.592+0000 7fe68effb640 1 -- 172.21.15.131:0/2030248608 >> v1:172.21.15.131:6790/0 conn(0x7fe688107e00 legacy=0x7fe68810a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:21.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.592+0000 7fe68effb640 1 -- 172.21.15.131:0/2030248608 shutdown_connections 2024-09-10T18:39:21.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.592+0000 7fe68effb640 1 -- 172.21.15.131:0/2030248608 >> 172.21.15.131:0/2030248608 conn(0x7fe6880752c0 msgr2=0x7fe6880f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:21.783 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.592+0000 7fe68effb640 1 -- 172.21.15.131:0/2030248608 shutdown_connections 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.592+0000 7fe68effb640 1 -- 172.21.15.131:0/2030248608 wait complete. 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.592+0000 7fe68effb640 1 Processor -- start 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe68effb640 1 -- start start 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe68effb640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe68811bf60 con 0x7fe6881042a0 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe68effb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe68811d120 con 0x7fe688101530 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe68effb640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe68811e320 con 0x7fe688107e00 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe687fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fe6881042a0 0x7fe688065720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:59668/0 (socket says 172.21.15.131:59668) 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe687fff640 1 -- 172.21.15.131:0/632692627 learned_addr learned my addr 172.21.15.131:0/632692627 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346611142 0 0) 0x7fe68811bf60 con 0x7fe6881042a0 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6740034e0 con 0x7fe6881042a0 2024-09-10T18:39:21.784 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513443553 0 0) 0x7fe68811e320 con 0x7fe688107e00 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe68811bf60 con 0x7fe688107e00 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025286305 0 0) 0x7fe68811d120 con 0x7fe688101530 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe68811e320 con 0x7fe688101530 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 254740754 0 0) 0x7fe6740034e0 con 0x7fe6881042a0 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe68811d120 con 0x7fe6881042a0 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe678001a70 con 0x7fe6881042a0 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.593+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2373826497 0 0) 0x7fe68811bf60 con 0x7fe688107e00 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe6740034e0 con 0x7fe688107e00 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe680002200 con 0x7fe688107e00 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2256521512 0 0) 0x7fe68811e320 con 0x7fe688101530 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe68811bf60 con 0x7fe688101530 2024-09-10T18:39:21.785 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe6700018c0 con 0x7fe688101530 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 251935530 0 0) 0x7fe68811d120 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 >> v1:172.21.15.131:6790/0 conn(0x7fe688107e00 legacy=0x7fe68811a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 >> v1:172.21.15.175:6789/0 conn(0x7fe688101530 legacy=0x7fe688065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68811f520 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68811e510 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68811e9e0 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fe678002390 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.594+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fe678005940 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.595+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.131:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe668000f20 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.595+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fe678002500 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.596+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fe678089cc0 con 0x7fe6881042a0 2024-09-10T18:39:21.786 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.596+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.175:6800/2653212041 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe67407e2e0 con 0x7fe67407a640 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.596+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7fe67808a0e0 con 0x7fe6881042a0 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.597+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== osd.4 v1:172.21.15.175:6800/2653212041 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fe67407e2e0 con 0x7fe67407a640 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.609+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 --> v1:172.21.15.175:6800/2653212041 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe668002b10 con 0x7fe67407a640 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.610+0000 7fe6857fa640 1 -- 172.21.15.131:0/632692627 <== osd.4 v1:172.21.15.175:6800/2653212041 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1156102027) 0x7fe668002b10 con 0x7fe67407a640 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.610+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 >> v1:172.21.15.175:6800/2653212041 conn(0x7fe67407a640 legacy=0x7fe67407ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.610+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 >> v1:172.21.15.131:6800/336444945 conn(0x7fe67406fdf0 legacy=0x7fe6740722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.610+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 >> v1:172.21.15.131:6789/0 conn(0x7fe6881042a0 legacy=0x7fe688065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:21.787 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.610+0000 7fe68d598640 1 -- 172.21.15.131:0/632692627 reap_dead start 2024-09-10T18:39:21.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.611+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 shutdown_connections 2024-09-10T18:39:21.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.611+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 >> 172.21.15.131:0/632692627 conn(0x7fe6880752c0 msgr2=0x7fe6880f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:21.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.611+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 shutdown_connections 2024-09-10T18:39:21.788 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:19.611+0000 7fe68effb640 1 -- 172.21.15.131:0/632692627 wait complete. 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.700+0000 7f1540bfb640 1 -- 172.21.15.131:0/3034843276 >> v1:172.21.15.131:6789/0 conn(0x7f153c0fd5d0 legacy=0x7f153c0fd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.700+0000 7f1540bfb640 1 -- 172.21.15.131:0/3034843276 shutdown_connections 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.700+0000 7f1540bfb640 1 -- 172.21.15.131:0/3034843276 >> 172.21.15.131:0/3034843276 conn(0x7f153c0752c0 msgr2=0x7f153c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.700+0000 7f1540bfb640 1 -- 172.21.15.131:0/3034843276 shutdown_connections 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.700+0000 7f1540bfb640 1 -- 172.21.15.131:0/3034843276 wait complete. 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1540bfb640 1 Processor -- start 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1540bfb640 1 -- start start 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1540bfb640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f153c11bf60 con 0x7f153c108f00 2024-09-10T18:39:22.060 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1540bfb640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f153c11d160 con 0x7f153c0fd5d0 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1540bfb640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f153c11e360 con 0x7f153c0fe1a0 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1531d9b640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f153c0fe1a0 0x7f153c1048b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:58724/0 (socket says 172.21.15.131:58724) 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f1531d9b640 1 -- 172.21.15.131:0/2231944871 learned_addr learned my addr 172.21.15.131:0/2231944871 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.701+0000 7f153ad9d640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f153c108f00 0x7f153c11a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56594/0 (socket says 172.21.15.131:56594) 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189467433 0 0) 0x7f153c11bf60 con 0x7f153c108f00 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15280034e0 con 0x7f153c108f00 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560603293 0 0) 0x7f153c11e360 con 0x7f153c0fe1a0 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f153c11bf60 con 0x7f153c0fe1a0 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921411411 0 0) 0x7f153c11d160 con 0x7f153c0fd5d0 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f153c11e360 con 0x7f153c0fd5d0 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 452307121 0 0) 0x7f15280034e0 con 0x7f153c108f00 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f153c11d160 con 0x7f153c108f00 2024-09-10T18:39:22.061 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f1534001bc0 con 0x7f153c108f00 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1656994188 0 0) 0x7f153c11bf60 con 0x7f153c0fe1a0 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f15280034e0 con 0x7f153c0fe1a0 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f1524001aa0 con 0x7f153c0fe1a0 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3061206460 0 0) 0x7f153c11e360 con 0x7f153c0fd5d0 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f153c11bf60 con 0x7f153c0fd5d0 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f152c001f00 con 0x7f153c0fd5d0 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2363545107 0 0) 0x7f153c11d160 con 0x7f153c108f00 2024-09-10T18:39:22.062 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 >> v1:172.21.15.131:6790/0 conn(0x7f153c0fe1a0 legacy=0x7f153c1048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 >> v1:172.21.15.175:6789/0 conn(0x7f153c0fd5d0 legacy=0x7f153c1041a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f153c11f560 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c11e550 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c11e9a0 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f1534002620 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.702+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f1534005ca0 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.703+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f151c000f20 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.704+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f15340021d0 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.704+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f1534089f80 con 0x7f153c108f00 2024-09-10T18:39:22.063 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.704+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6805/2444922616 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f152807e300 con 0x7f152807a660 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.704+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f153408a3a0 con 0x7f153c108f00 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.705+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== osd.1 v1:172.21.15.131:6805/2444922616 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f152807e300 con 0x7f152807a660 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.718+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 --> v1:172.21.15.131:6805/2444922616 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f151c002b10 con 0x7f152807a660 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.718+0000 7f15337fe640 1 -- 172.21.15.131:0/2231944871 <== osd.1 v1:172.21.15.131:6805/2444922616 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3910016750) 0x7f151c002b10 con 0x7f152807a660 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 >> v1:172.21.15.131:6805/2444922616 conn(0x7f152807a660 legacy=0x7f152807ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 >> v1:172.21.15.131:6800/336444945 conn(0x7f152806ff90 legacy=0x7f1528072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 >> v1:172.21.15.131:6789/0 conn(0x7f153c108f00 legacy=0x7f153c11a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f153ad9d640 1 -- 172.21.15.131:0/2231944871 reap_dead start 2024-09-10T18:39:22.064 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 shutdown_connections 2024-09-10T18:39:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 >> 172.21.15.131:0/2231944871 conn(0x7f153c0752c0 msgr2=0x7f153c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 shutdown_connections 2024-09-10T18:39:22.065 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.719+0000 7f1540bfb640 1 -- 172.21.15.131:0/2231944871 wait complete. 2024-09-10T18:39:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 -- 172.21.15.131:0/1157456594 >> v1:172.21.15.131:6789/0 conn(0x7f8578107db0 legacy=0x7f857810a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 -- 172.21.15.131:0/1157456594 shutdown_connections 2024-09-10T18:39:22.160 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 -- 172.21.15.131:0/1157456594 >> 172.21.15.131:0/1157456594 conn(0x7f85780f7020 msgr2=0x7f85780f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 -- 172.21.15.131:0/1157456594 shutdown_connections 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 -- 172.21.15.131:0/1157456594 wait complete. 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 Processor -- start 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.603+0000 7f857ec20640 1 -- start start 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f857ec20640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f857811bef0 con 0x7f85781014e0 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f857ec20640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f857811d0f0 con 0x7f8578107db0 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f857ec20640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f857811e2f0 con 0x7f8578104250 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f857c9bc640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f85781014e0 0x7f8578065d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:56580/0 (socket says 172.21.15.131:56580) 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f857c9bc640 1 -- 172.21.15.131:0/334077714 learned_addr learned my addr 172.21.15.131:0/334077714 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818124117 0 0) 0x7f857811bef0 con 0x7f85781014e0 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85680034e0 con 0x7f85781014e0 2024-09-10T18:39:22.161 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535712657 0 0) 0x7f857811e2f0 con 0x7f8578104250 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f857811bef0 con 0x7f8578104250 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752445156 0 0) 0x7f857811d0f0 con 0x7f8578107db0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f857811e2f0 con 0x7f8578107db0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4291960990 0 0) 0x7f85680034e0 con 0x7f85781014e0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f857811d0f0 con 0x7f85781014e0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f856c001980 con 0x7f85781014e0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1546133627 0 0) 0x7f857811bef0 con 0x7f8578104250 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f85680034e0 con 0x7f8578104250 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.604+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f8564001830 con 0x7f8578104250 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2218544246 0 0) 0x7f857811e2f0 con 0x7f8578107db0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f857811bef0 con 0x7f8578107db0 2024-09-10T18:39:22.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f85700017b0 con 0x7f8578107db0 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3002066196 0 0) 0x7f857811d0f0 con 0x7f85781014e0 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 >> v1:172.21.15.131:6790/0 conn(0x7f8578104250 legacy=0x7f8578066410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 >> v1:172.21.15.175:6789/0 conn(0x7f8578107db0 legacy=0x7f857811a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857811f4f0 con 0x7f85781014e0 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f857811e4e0 con 0x7f85781014e0 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f856c0020b0 con 0x7f85781014e0 2024-09-10T18:39:22.163 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f856c005a40 con 0x7f85781014e0 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.605+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f857811e9b0 con 0x7f85781014e0 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.606+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.131:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f857811eee0 con 0x7f85781014e0 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.606+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f856c01bfb0 con 0x7f85781014e0 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.607+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f856c089fa0 con 0x7f85781014e0 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.607+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.175:6804/210062365 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f856807e320 con 0x7f856807a680 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.607+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f856c08a3d0 con 0x7f85781014e0 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.608+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== osd.5 v1:172.21.15.175:6804/210062365 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f856807e320 con 0x7f856807a680 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.620+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 --> v1:172.21.15.175:6804/210062365 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85780fcbe0 con 0x7f856807a680 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f85757fa640 1 -- 172.21.15.131:0/334077714 <== osd.5 v1:172.21.15.175:6804/210062365 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 291186429) 0x7f85780fcbe0 con 0x7f856807a680 2024-09-10T18:39:22.164 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 >> v1:172.21.15.175:6804/210062365 conn(0x7f856807a680 legacy=0x7f856807cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 >> v1:172.21.15.131:6800/336444945 conn(0x7f856806fe80 legacy=0x7f8568072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 >> v1:172.21.15.131:6789/0 conn(0x7f85781014e0 legacy=0x7f8578065d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857d1bd640 1 -- 172.21.15.131:0/334077714 reap_dead start 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 shutdown_connections 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 >> 172.21.15.131:0/334077714 conn(0x7f85780f7020 msgr2=0x7f8578064bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 shutdown_connections 2024-09-10T18:39:22.165 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:18.621+0000 7f857ec20640 1 -- 172.21.15.131:0/334077714 wait complete. 2024-09-10T18:39:22.737 INFO:teuthology.orchestra.run.smithi131.stdout:154618822681 2024-09-10T18:39:22.737 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:39:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:23 smithi131 ceph-mon[29369]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:23 smithi131 ceph-mon[40885]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:23 smithi175 ceph-mon[33399]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:23.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:39:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:39:23.973 INFO:teuthology.orchestra.run.smithi131.stdout:55834574891 2024-09-10T18:39:23.974 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:39:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:24.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:24.751 INFO:teuthology.orchestra.run.smithi131.stdout:107374182433 2024-09-10T18:39:24.752 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:39:25.333 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:39:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:25.404 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:25 smithi131 ceph-mon[29369]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:25.404 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:25 smithi131 ceph-mon[40885]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 -- 172.21.15.131:0/3414812849 >> v1:172.21.15.131:6790/0 conn(0x7f45a4105320 legacy=0x7f45a4107700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 -- 172.21.15.131:0/3414812849 shutdown_connections 2024-09-10T18:39:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 -- 172.21.15.131:0/3414812849 >> 172.21.15.131:0/3414812849 conn(0x7f45a40f8140 msgr2=0x7f45a40fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:25.434 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 -- 172.21.15.131:0/3414812849 shutdown_connections 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 -- 172.21.15.131:0/3414812849 wait complete. 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 Processor -- start 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.402+0000 7f45ac730640 1 -- start start 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f45ac730640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45a411bd10 con 0x7f45a4108eb0 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f45ac730640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45a411cef0 con 0x7f45a4105320 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f45ac730640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45a411e0f0 con 0x7f45a4102560 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f45aa4cc640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f45a4102560 0x7f45a40fe130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:41216/0 (socket says 172.21.15.131:41216) 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f45aa4cc640 1 -- 172.21.15.131:0/2669337760 learned_addr learned my addr 172.21.15.131:0/2669337760 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066364303 0 0) 0x7f45a411e0f0 con 0x7f45a4102560 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45900034e0 con 0x7f45a4102560 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807170547 0 0) 0x7f45a411bd10 con 0x7f45a4108eb0 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a411e0f0 con 0x7f45a4108eb0 2024-09-10T18:39:25.435 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992268433 0 0) 0x7f45a411cef0 con 0x7f45a4105320 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a411bd10 con 0x7f45a4105320 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1094868050 0 0) 0x7f45900034e0 con 0x7f45a4102560 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45a411cef0 con 0x7f45a4102560 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f4598001990 con 0x7f45a4102560 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3041493700 0 0) 0x7f45a411e0f0 con 0x7f45a4108eb0 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45900034e0 con 0x7f45a4108eb0 2024-09-10T18:39:25.436 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f459c0019a0 con 0x7f45a4108eb0 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2150901944 0 0) 0x7f45a411bd10 con 0x7f45a4105320 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45a411e0f0 con 0x7f45a4105320 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f45a0002020 con 0x7f45a4105320 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2436898472 0 0) 0x7f45a411cef0 con 0x7f45a4102560 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 >> v1:172.21.15.175:6789/0 conn(0x7f45a4105320 legacy=0x7f45a40fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 >> v1:172.21.15.131:6789/0 conn(0x7f45a4108eb0 legacy=0x7f45a411a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a411f2f0 con 0x7f45a4102560 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.403+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a411bf00 con 0x7f45a4102560 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.404+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a411c350 con 0x7f45a4102560 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.404+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f4598001d40 con 0x7f45a4102560 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.404+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f4598005bd0 con 0x7f45a4102560 2024-09-10T18:39:25.437 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.404+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.131:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4588000f50 con 0x7f45a4102560 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.405+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f459801c070 con 0x7f45a4102560 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.406+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f4598089ef0 con 0x7f45a4102560 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.406+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.175:6808/3246498802 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f459007e0e0 con 0x7f459007a440 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.406+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f459808a320 con 0x7f45a4102560 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.407+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== osd.6 v1:172.21.15.175:6808/3246498802 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f459007e0e0 con 0x7f459007a440 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.419+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 --> v1:172.21.15.175:6808/3246498802 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4588002b40 con 0x7f459007a440 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f4596ffd640 1 -- 172.21.15.131:0/2669337760 <== osd.6 v1:172.21.15.175:6808/3246498802 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1654995110) 0x7f4588002b40 con 0x7f459007a440 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 >> v1:172.21.15.175:6808/3246498802 conn(0x7f459007a440 legacy=0x7f459007c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 >> v1:172.21.15.131:6800/336444945 conn(0x7f459006fe60 legacy=0x7f4590072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 >> v1:172.21.15.131:6790/0 conn(0x7f45a4102560 legacy=0x7f45a40fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45aaccd640 1 -- 172.21.15.131:0/2669337760 reap_dead start 2024-09-10T18:39:25.438 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 shutdown_connections 2024-09-10T18:39:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 >> 172.21.15.131:0/2669337760 conn(0x7f45a40f8140 msgr2=0x7f45a410bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 shutdown_connections 2024-09-10T18:39:25.439 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:25.420+0000 7f45ac730640 1 -- 172.21.15.131:0/2669337760 wait complete. 2024-09-10T18:39:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:25 smithi175 ceph-mon[33399]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:26.999 INFO:teuthology.orchestra.run.smithi131.stdout:180388626453 2024-09-10T18:39:26.999 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:39:27.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.429+0000 7fa750f77640 1 -- 172.21.15.131:0/462507796 >> v1:172.21.15.131:6790/0 conn(0x7fa74c104270 legacy=0x7fa74c106650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:27.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.429+0000 7fa750f77640 1 -- 172.21.15.131:0/462507796 shutdown_connections 2024-09-10T18:39:27.444 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.429+0000 7fa750f77640 1 -- 172.21.15.131:0/462507796 >> 172.21.15.131:0/462507796 conn(0x7fa74c0f7000 msgr2=0x7fa74c0f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.429+0000 7fa750f77640 1 -- 172.21.15.131:0/462507796 shutdown_connections 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.429+0000 7fa750f77640 1 -- 172.21.15.131:0/462507796 wait complete. 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa750f77640 1 Processor -- start 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa750f77640 1 -- start start 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa750f77640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa74c11add0 con 0x7fa74c107dd0 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa750f77640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa74c11bf90 con 0x7fa74c104270 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa750f77640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa74c11d190 con 0x7fa74c101500 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa74a59c640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fa74c101500 0x7fa74c104640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:41226/0 (socket says 172.21.15.131:41226) 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa74a59c640 1 -- 172.21.15.131:0/1365413474 learned_addr learned my addr 172.21.15.131:0/1365413474 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.430+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253092659 0 0) 0x7fa74c11d190 con 0x7fa74c101500 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa738003120 con 0x7fa74c101500 2024-09-10T18:39:27.445 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077638034 0 0) 0x7fa74c11add0 con 0x7fa74c107dd0 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa74c11d190 con 0x7fa74c107dd0 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1525957135 0 0) 0x7fa74c11bf90 con 0x7fa74c104270 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa74c11add0 con 0x7fa74c104270 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3382873613 0 0) 0x7fa738003120 con 0x7fa74c101500 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa74c11bf90 con 0x7fa74c101500 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa73c0019f0 con 0x7fa74c101500 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 753462807 0 0) 0x7fa74c11add0 con 0x7fa74c104270 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa738003120 con 0x7fa74c104270 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1748804384 0 0) 0x7fa74c11d190 con 0x7fa74c107dd0 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa74c11add0 con 0x7fa74c107dd0 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa734001a00 con 0x7fa74c104270 2024-09-10T18:39:27.446 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa744001b80 con 0x7fa74c107dd0 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4067211555 0 0) 0x7fa74c11bf90 con 0x7fa74c101500 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 >> v1:172.21.15.175:6789/0 conn(0x7fa74c104270 legacy=0x7fa74c104db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 >> v1:172.21.15.131:6789/0 conn(0x7fa74c107dd0 legacy=0x7fa74c1194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c11e390 con 0x7fa74c101500 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c11afc0 con 0x7fa74c101500 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c11b490 con 0x7fa74c101500 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fa73c0023e0 con 0x7fa74c101500 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.431+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fa73c005a40 con 0x7fa74c101500 2024-09-10T18:39:27.447 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.432+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa72c000f20 con 0x7fa74c101500 2024-09-10T18:39:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.433+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fa73c002550 con 0x7fa74c101500 2024-09-10T18:39:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.433+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fa73c089f70 con 0x7fa74c101500 2024-09-10T18:39:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.433+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6809/1436967876 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa73807e400 con 0x7fa73807a800 2024-09-10T18:39:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.433+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7fa73c08a390 con 0x7fa74c101500 2024-09-10T18:39:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.434+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== osd.2 v1:172.21.15.131:6809/1436967876 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7fa73807e400 con 0x7fa73807a800 2024-09-10T18:39:27.448 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.446+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 --> v1:172.21.15.131:6809/1436967876 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa72c002b10 con 0x7fa73807a800 2024-09-10T18:39:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa742ffd640 1 -- 172.21.15.131:0/1365413474 <== osd.2 v1:172.21.15.131:6809/1436967876 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1016321414) 0x7fa72c002b10 con 0x7fa73807a800 2024-09-10T18:39:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 >> v1:172.21.15.131:6809/1436967876 conn(0x7fa73807a800 legacy=0x7fa73807cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 >> v1:172.21.15.131:6800/336444945 conn(0x7fa73806fef0 legacy=0x7fa7380723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:27.449 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 >> v1:172.21.15.131:6790/0 conn(0x7fa74c101500 legacy=0x7fa74c104640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa74ad9d640 1 -- 172.21.15.131:0/1365413474 reap_dead start 2024-09-10T18:39:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 shutdown_connections 2024-09-10T18:39:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 >> 172.21.15.131:0/1365413474 conn(0x7fa74c0f7000 msgr2=0x7fa74c0fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 shutdown_connections 2024-09-10T18:39:27.450 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:27.447+0000 7fa750f77640 1 -- 172.21.15.131:0/1365413474 wait complete. 2024-09-10T18:39:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:27 smithi175 ceph-mon[33399]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:27 smithi131 ceph-mon[29369]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:27 smithi131 ceph-mon[40885]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:28.751 INFO:teuthology.orchestra.run.smithi131.stdout:128849018910 2024-09-10T18:39:28.751 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:39:28.865 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:29.093 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:29.094 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:29.569 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:29 smithi131 ceph-mon[29369]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:29.569 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:29 smithi131 ceph-mon[40885]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:29 smithi175 ceph-mon[33399]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:29.717 INFO:teuthology.orchestra.run.smithi131.stdout:77309411368 2024-09-10T18:39:29.717 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.800+0000 7f3075573640 1 -- 172.21.15.131:0/3536114396 >> v1:172.21.15.131:6789/0 conn(0x7f3070104260 legacy=0x7f3070106640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.800+0000 7f3075573640 1 -- 172.21.15.131:0/3536114396 shutdown_connections 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.800+0000 7f3075573640 1 -- 172.21.15.131:0/3536114396 >> 172.21.15.131:0/3536114396 conn(0x7f30700f7050 msgr2=0x7f30700f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.800+0000 7f3075573640 1 -- 172.21.15.131:0/3536114396 shutdown_connections 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.800+0000 7f3075573640 1 -- 172.21.15.131:0/3536114396 wait complete. 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.801+0000 7f3075573640 1 Processor -- start 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.801+0000 7f3075573640 1 -- start start 2024-09-10T18:39:30.807 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.801+0000 7f3075573640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f307011c010 con 0x7f3070104260 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f3075573640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f307011d210 con 0x7f30701014f0 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f3075573640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f307011e410 con 0x7f3070107dc0 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f306e7fc640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3070104260 0x7f3070105a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51960/0 (socket says 172.21.15.131:51960) 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f306e7fc640 1 -- 172.21.15.131:0/3045969687 learned_addr learned my addr 172.21.15.131:0/3045969687 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f306f7fe640 1 --1- 172.21.15.131:0/3045969687 >> v1:172.21.15.131:6790/0 conn(0x7f3070107dc0 0x7f307011a710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:45060/0 (socket says 172.21.15.131:45060) 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224898443 0 0) 0x7f307011e410 con 0x7f3070107dc0 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f305c003200 con 0x7f3070107dc0 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545797467 0 0) 0x7f307011c010 con 0x7f3070104260 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f307011e410 con 0x7f3070104260 2024-09-10T18:39:30.808 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747354975 0 0) 0x7f307011d210 con 0x7f30701014f0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f307011c010 con 0x7f30701014f0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2963678378 0 0) 0x7f305c003200 con 0x7f3070107dc0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f307011d210 con 0x7f3070107dc0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3068001bc0 con 0x7f3070107dc0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2162590011 0 0) 0x7f307011e410 con 0x7f3070104260 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f305c003200 con 0x7f3070104260 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3011062374 0 0) 0x7f307011c010 con 0x7f30701014f0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f307011e410 con 0x7f30701014f0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3058002a60 con 0x7f3070104260 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3060001620 con 0x7f30701014f0 2024-09-10T18:39:30.809 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1700919838 0 0) 0x7f307011d210 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 >> v1:172.21.15.175:6789/0 conn(0x7f30701014f0 legacy=0x7f3070105320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 >> v1:172.21.15.131:6789/0 conn(0x7f3070104260 legacy=0x7f3070105a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307011f610 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.802+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307011e600 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.803+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f307011ea50 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.803+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f3068002640 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.803+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3068005d20 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.803+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.131:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3050000f20 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.804+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f30680021f0 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.805+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f3068089f30 con 0x7f3070107dc0 2024-09-10T18:39:30.810 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.805+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.175:6812/2356238523 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f305c07e4b0 con 0x7f305c07a810 2024-09-10T18:39:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.805+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (unknown 1962166362 0 0) 0x7f306808a360 con 0x7f3070107dc0 2024-09-10T18:39:30.811 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.806+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== osd.7 v1:172.21.15.175:6812/2356238523 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (unknown 0 0 1173804961) 0x7f305c07e4b0 con 0x7f305c07a810 2024-09-10T18:39:30.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.818+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 --> v1:172.21.15.175:6812/2356238523 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3050002b10 con 0x7f305c07a810 2024-09-10T18:39:30.820 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f30677fe640 1 -- 172.21.15.131:0/3045969687 <== osd.7 v1:172.21.15.175:6812/2356238523 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1074458960) 0x7f3050002b10 con 0x7f305c07a810 2024-09-10T18:39:30.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 >> v1:172.21.15.175:6812/2356238523 conn(0x7f305c07a810 legacy=0x7f305c07cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:30.821 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 >> v1:172.21.15.131:6800/336444945 conn(0x7f305c06fe80 legacy=0x7f305c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 >> v1:172.21.15.131:6790/0 conn(0x7f3070107dc0 legacy=0x7f307011a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f306f7fe640 1 -- 172.21.15.131:0/3045969687 reap_dead start 2024-09-10T18:39:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 shutdown_connections 2024-09-10T18:39:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 >> 172.21.15.131:0/3045969687 conn(0x7f30700f7050 msgr2=0x7f30700fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 shutdown_connections 2024-09-10T18:39:30.822 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:30.819+0000 7f3075573640 1 -- 172.21.15.131:0/3045969687 wait complete. 2024-09-10T18:39:31.435 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:31 smithi131 ceph-mon[29369]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:31.435 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:31 smithi131 ceph-mon[40885]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:31.460 INFO:teuthology.orchestra.run.smithi131.stdout:206158430225 2024-09-10T18:39:31.461 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:39:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:31 smithi175 ceph-mon[33399]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:32.079 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:32.924 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:32 smithi131 podman[75710]: 2024-09-10 18:39:32.734070341 +0000 UTC m=+1.057400841 container create b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:39:33.601 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-mon[29369]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:39:33.602 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-mon[40885]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:39:33.602 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 podman[75710]: 2024-09-10 18:39:33.390020309 +0000 UTC m=+1.713350809 container init b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:39:33.602 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 podman[75710]: 2024-09-10 18:39:33.395170581 +0000 UTC m=+1.718501089 container start b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:39:33.602 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.408Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:39:33.602 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.408Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:39:33.602 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.409Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.131 port=9094 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.410Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.432Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.433Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.435Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:33.435Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 bash[75710]: b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 2024-09-10T18:39:33.603 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:33 smithi131 systemd[1]: Started Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:39:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:33 smithi175 ceph-mon[33399]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:39:33.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:39:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:39:33.914 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:33] ENGINE Bus STOPPING 2024-09-10T18:39:34.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:39:34.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:33] ENGINE Bus STOPPED 2024-09-10T18:39:34.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:33] ENGINE Bus STARTING 2024-09-10T18:39:34.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:34] ENGINE Serving on http://:::9283 2024-09-10T18:39:34.328 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:39:34] ENGINE Bus STARTED 2024-09-10T18:39:34.751 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:34.751 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.751 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.752 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.753 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.753 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.753 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:39:34.753 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:34.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:39:34.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:35.428 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:39:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:35.736 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:35.737 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:35.821 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:39:35.822 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-mon[29369]: Deploying daemon grafana.a on smithi175 2024-09-10T18:39:35.822 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-mon[29369]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:35.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:39:35.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-mon[40885]: Deploying daemon grafana.a on smithi175 2024-09-10T18:39:35.822 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-mon[40885]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:35.822 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:35 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:35.410Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000152585s 2024-09-10T18:39:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:35 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:39:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:35 smithi175 ceph-mon[33399]: Deploying daemon grafana.a on smithi175 2024-09-10T18:39:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:35 smithi175 ceph-mon[33399]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 -- 172.21.15.131:0/574453732 >> v1:172.21.15.131:6789/0 conn(0x7f19e4108ed0 legacy=0x7f19e410b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 -- 172.21.15.131:0/574453732 shutdown_connections 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 -- 172.21.15.131:0/574453732 >> 172.21.15.131:0/574453732 conn(0x7f19e40f7000 msgr2=0x7f19e40f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 -- 172.21.15.131:0/574453732 shutdown_connections 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 -- 172.21.15.131:0/574453732 wait complete. 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 Processor -- start 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.331+0000 7f19ea363640 1 -- start start 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19ea363640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19e411c0c0 con 0x7f19e40fd5a0 2024-09-10T18:39:36.360 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19ea363640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19e411d2c0 con 0x7f19e40fe170 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19ea363640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19e411e4c0 con 0x7f19e4108ed0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e3fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f19e40fd5a0 0x7f19e4104290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51982/0 (socket says 172.21.15.131:51982) 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e3fff640 1 -- 172.21.15.131:0/1450678382 learned_addr learned my addr 172.21.15.131:0/1450678382 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697992425 0 0) 0x7f19e411e4c0 con 0x7f19e4108ed0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19cc0034e0 con 0x7f19e4108ed0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134223017 0 0) 0x7f19e411c0c0 con 0x7f19e40fd5a0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e411e4c0 con 0x7f19e40fd5a0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3148632893 0 0) 0x7f19e411d2c0 con 0x7f19e40fe170 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e411c0c0 con 0x7f19e40fe170 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3065874768 0 0) 0x7f19cc0034e0 con 0x7f19e4108ed0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19e411d2c0 con 0x7f19e4108ed0 2024-09-10T18:39:36.361 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f19dc002f30 con 0x7f19e4108ed0 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3698679758 0 0) 0x7f19e411e4c0 con 0x7f19e40fd5a0 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19cc0034e0 con 0x7f19e40fd5a0 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f19d40018e0 con 0x7f19e40fd5a0 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2900284974 0 0) 0x7f19e411c0c0 con 0x7f19e40fe170 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19e411e4c0 con 0x7f19e40fe170 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f19d8001840 con 0x7f19e40fe170 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4022947657 0 0) 0x7f19e411d2c0 con 0x7f19e4108ed0 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 >> v1:172.21.15.175:6789/0 conn(0x7f19e40fe170 legacy=0x7f19e41049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.362 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 >> v1:172.21.15.131:6789/0 conn(0x7f19e40fd5a0 legacy=0x7f19e4104290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e411f6c0 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e411e6b0 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f19dc001eb0 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.332+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e411eb00 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.333+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f19dc003850 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.334+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d0005010 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.334+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f19dc019d80 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.334+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f19dc087970 con 0x7f19e4108ed0 2024-09-10T18:39:36.363 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.339+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f19dc054660 con 0x7f19e4108ed0 2024-09-10T18:39:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.500+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f19d00052c0 con 0x7f19e4108ed0 2024-09-10T18:39:36.502 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.500+0000 7f19e0ff9640 1 -- 172.21.15.131:0/1450678382 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 2057152787) 0x7f19dc054000 con 0x7f19e4108ed0 2024-09-10T18:39:36.502 INFO:teuthology.orchestra.run.smithi131.stdout:154618822684 2024-09-10T18:39:36.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.502+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 >> v1:172.21.15.131:6800/336444945 conn(0x7f19cc06fe10 legacy=0x7f19cc0722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.502+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 >> v1:172.21.15.131:6790/0 conn(0x7f19e4108ed0 legacy=0x7f19e411a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.502+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 shutdown_connections 2024-09-10T18:39:36.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.502+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 >> 172.21.15.131:0/1450678382 conn(0x7f19e40f7000 msgr2=0x7f19e40f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:36.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.502+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 shutdown_connections 2024-09-10T18:39:36.504 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.502+0000 7f19ea363640 1 -- 172.21.15.131:0/1450678382 wait complete. 2024-09-10T18:39:36.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.547+0000 7fdd78a18640 1 -- 172.21.15.131:0/1341671305 >> v1:172.21.15.131:6790/0 conn(0x7fdd74101530 legacy=0x7fdd74101900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.547+0000 7fdd78a18640 1 -- 172.21.15.131:0/1341671305 shutdown_connections 2024-09-10T18:39:36.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.547+0000 7fdd78a18640 1 -- 172.21.15.131:0/1341671305 >> 172.21.15.131:0/1341671305 conn(0x7fdd740752c0 msgr2=0x7fdd740f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:36.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.547+0000 7fdd78a18640 1 -- 172.21.15.131:0/1341671305 shutdown_connections 2024-09-10T18:39:36.569 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.547+0000 7fdd78a18640 1 -- 172.21.15.131:0/1341671305 wait complete. 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.547+0000 7fdd78a18640 1 Processor -- start 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd78a18640 1 -- start start 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd78a18640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdd7411ad50 con 0x7fdd741042a0 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd78a18640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdd7411bf50 con 0x7fdd74107e00 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd78a18640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdd7411d150 con 0x7fdd74101530 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd7259c640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fdd74101530 0x7fdd74104670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:45100/0 (socket says 172.21.15.131:45100) 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd7259c640 1 -- 172.21.15.131:0/2396336608 learned_addr learned my addr 172.21.15.131:0/2396336608 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723003389 0 0) 0x7fdd7411ad50 con 0x7fdd741042a0 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd60003120 con 0x7fdd741042a0 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775844938 0 0) 0x7fdd7411d150 con 0x7fdd74101530 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7411ad50 con 0x7fdd74101530 2024-09-10T18:39:36.570 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439237595 0 0) 0x7fdd7411bf50 con 0x7fdd74107e00 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd7411d150 con 0x7fdd74107e00 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1140302954 0 0) 0x7fdd7411ad50 con 0x7fdd74101530 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd7411bf50 con 0x7fdd74101530 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdd64001a00 con 0x7fdd74101530 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3239214051 0 0) 0x7fdd60003120 con 0x7fdd741042a0 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd7411ad50 con 0x7fdd741042a0 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdd5c001620 con 0x7fdd741042a0 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3082695912 0 0) 0x7fdd7411d150 con 0x7fdd74107e00 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdd60003120 con 0x7fdd74107e00 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.548+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdd6c001c00 con 0x7fdd74107e00 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1977979216 0 0) 0x7fdd7411bf50 con 0x7fdd74101530 2024-09-10T18:39:36.571 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 >> v1:172.21.15.175:6789/0 conn(0x7fdd74107e00 legacy=0x7fdd74119450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 >> v1:172.21.15.131:6789/0 conn(0x7fdd741042a0 legacy=0x7fdd74104d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7411e350 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7411d340 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7411d810 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fdd640014d0 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.549+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdd64003710 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.550+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd54005010 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.550+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fdd64003020 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.551+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fdd6408a080 con 0x7fdd74101530 2024-09-10T18:39:36.572 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.555+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdd64056dd0 con 0x7fdd74101530 2024-09-10T18:39:36.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.716+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdd540052c0 con 0x7fdd74101530 2024-09-10T18:39:36.718 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.717+0000 7fdd6b7fe640 1 -- 172.21.15.131:0/2396336608 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 2037106864) 0x7fdd64056770 con 0x7fdd74101530 2024-09-10T18:39:36.719 INFO:teuthology.orchestra.run.smithi131.stdout:55834574894 2024-09-10T18:39:36.719 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1450678382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:39:36.719 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1450678382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:39:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.719+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 >> v1:172.21.15.131:6800/336444945 conn(0x7fdd6006fea0 legacy=0x7fdd60072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.719+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 >> v1:172.21.15.131:6790/0 conn(0x7fdd74101530 legacy=0x7fdd74104670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.719+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 shutdown_connections 2024-09-10T18:39:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.719+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 >> 172.21.15.131:0/2396336608 conn(0x7fdd740752c0 msgr2=0x7fdd740646d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.719+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 shutdown_connections 2024-09-10T18:39:36.721 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:36.719+0000 7fdd78a18640 1 -- 172.21.15.131:0/2396336608 wait complete. 2024-09-10T18:39:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1450678382' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:39:37.131 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822681 got 154618822684 for osd.5 2024-09-10T18:39:37.131 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:37.548 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:37.660 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:37 smithi131 ceph-mon[29369]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:37.661 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2396336608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:39:37.661 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:37 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:37.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:37 smithi131 ceph-mon[40885]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:37.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2396336608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:39:37.661 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:37 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:37 smithi175 ceph-mon[33399]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2396336608' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:39:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:37 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:39.799 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:39 smithi131 ceph-mon[29369]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:39.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:39 smithi131 ceph-mon[40885]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:39.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574894 for osd.1 2024-09-10T18:39:39.872 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:39 smithi175 ceph-mon[33399]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1174334887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:39:41.026 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1174334887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:39:41.026 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1174334887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:39:41.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.620+0000 7fdfe3294640 1 -- 172.21.15.131:0/4249573934 >> v1:172.21.15.131:6789/0 conn(0x7fdfdc108eb0 legacy=0x7fdfdc10b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:41.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.620+0000 7fdfe3294640 1 -- 172.21.15.131:0/4249573934 shutdown_connections 2024-09-10T18:39:41.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.620+0000 7fdfe3294640 1 -- 172.21.15.131:0/4249573934 >> 172.21.15.131:0/4249573934 conn(0x7fdfdc0f7020 msgr2=0x7fdfdc0f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:41.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.620+0000 7fdfe3294640 1 -- 172.21.15.131:0/4249573934 shutdown_connections 2024-09-10T18:39:41.176 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.620+0000 7fdfe3294640 1 -- 172.21.15.131:0/4249573934 wait complete. 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.620+0000 7fdfe3294640 1 Processor -- start 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe3294640 1 -- start start 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe3294640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdfdc11bf60 con 0x7fdfdc108eb0 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe3294640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdfdc11d120 con 0x7fdfdc0fe150 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe3294640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdfdc11e320 con 0x7fdfdc0fd580 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe1030640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fdfdc0fd580 0x7fdfdc1040f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:49070/0 (socket says 172.21.15.131:49070) 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe1030640 1 -- 172.21.15.131:0/1174334887 learned_addr learned my addr 172.21.15.131:0/1174334887 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfe1831640 1 --1- 172.21.15.131:0/1174334887 >> v1:172.21.15.131:6789/0 conn(0x7fdfdc108eb0 0x7fdfdc11a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37300/0 (socket says 172.21.15.131:37300) 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790931449 0 0) 0x7fdfdc11bf60 con 0x7fdfdc108eb0 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfc8003120 con 0x7fdfdc108eb0 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226949081 0 0) 0x7fdfdc11e320 con 0x7fdfdc0fd580 2024-09-10T18:39:41.177 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfdc11bf60 con 0x7fdfdc0fd580 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093669738 0 0) 0x7fdfdc11d120 con 0x7fdfdc0fe150 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfdc11e320 con 0x7fdfdc0fe150 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1317868215 0 0) 0x7fdfdc11bf60 con 0x7fdfdc0fd580 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfdc11d120 con 0x7fdfdc0fd580 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2780724539 0 0) 0x7fdfc8003120 con 0x7fdfdc108eb0 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfdc11bf60 con 0x7fdfdc108eb0 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdfcc001a70 con 0x7fdfdc0fd580 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 201818225 0 0) 0x7fdfdc11e320 con 0x7fdfdc0fe150 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdfc8003120 con 0x7fdfdc0fe150 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.621+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdfd4002ed0 con 0x7fdfdc108eb0 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdfd80017c0 con 0x7fdfdc0fe150 2024-09-10T18:39:41.178 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2217473653 0 0) 0x7fdfdc11d120 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 >> v1:172.21.15.175:6789/0 conn(0x7fdfdc0fe150 legacy=0x7fdfdc104800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 >> v1:172.21.15.131:6789/0 conn(0x7fdfdc108eb0 legacy=0x7fdfdc11a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc11f520 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfdc11e510 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fdfcc002410 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdfcc005980 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.622+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfdc11e960 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.623+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfdc0663c0 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.623+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fdfcc002580 con 0x7fdfdc0fd580 2024-09-10T18:39:41.179 INFO:teuthology.orchestra.run.smithi131.stdout:34359738420 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.624+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fdfcc089f20 con 0x7fdfdc0fd580 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.628+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdfcc0567f0 con 0x7fdfdc0fd580 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.785+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdfdc04bf60 con 0x7fdfdc0fd580 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.786+0000 7fdfd1ffb640 1 -- 172.21.15.131:0/1174334887 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 418653114) 0x7fdfcc056610 con 0x7fdfdc0fd580 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.787+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 >> v1:172.21.15.131:6800/336444945 conn(0x7fdfc806fdf0 legacy=0x7fdfc80722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.787+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 >> v1:172.21.15.131:6790/0 conn(0x7fdfdc0fd580 legacy=0x7fdfdc1040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.788+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 shutdown_connections 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.788+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 >> 172.21.15.131:0/1174334887 conn(0x7fdfdc0f7020 msgr2=0x7fdfdc0f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.788+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 shutdown_connections 2024-09-10T18:39:41.180 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:39.788+0000 7fdfe3294640 1 -- 172.21.15.131:0/1174334887 wait complete. 2024-09-10T18:39:41.776 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:41 smithi131 ceph-mon[40885]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:41.776 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:41 smithi131 ceph-mon[29369]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:41.805 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738416 got 34359738420 for osd.0 2024-09-10T18:39:41.806 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:41.908 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:41 smithi175 ceph-mon[33399]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1182903247 >> v1:172.21.15.131:6789/0 conn(0x7ff828105340 legacy=0x7ff828107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1182903247 shutdown_connections 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1182903247 >> 172.21.15.131:0/1182903247 conn(0x7ff8280f8140 msgr2=0x7ff8280fa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1182903247 shutdown_connections 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1182903247 wait complete. 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 Processor -- start 2024-09-10T18:39:42.593 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.560+0000 7ff82ec5f640 1 -- start start 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82ec5f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff82811bdd0 con 0x7ff828105340 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82ec5f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff82811cfb0 con 0x7ff828102580 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82ec5f640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff82811e1b0 con 0x7ff828108ed0 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82d1fc640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7ff828108ed0 0x7ff82811a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:49088/0 (socket says 172.21.15.131:49088) 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82d1fc640 1 -- 172.21.15.131:0/1735531885 learned_addr learned my addr 172.21.15.131:0/1735531885 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81ffff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff828105340 0x7ff8280fe840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37328/0 (socket says 172.21.15.131:37328) 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150110577 0 0) 0x7ff82811e1b0 con 0x7ff828108ed0 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8100034e0 con 0x7ff828108ed0 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192064763 0 0) 0x7ff82811bdd0 con 0x7ff828105340 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82811e1b0 con 0x7ff828105340 2024-09-10T18:39:42.594 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729015610 0 0) 0x7ff82811cfb0 con 0x7ff828102580 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff82811bdd0 con 0x7ff828102580 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 14265815 0 0) 0x7ff8100034e0 con 0x7ff828108ed0 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff82811cfb0 con 0x7ff828108ed0 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff820001cd0 con 0x7ff828108ed0 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3189116288 0 0) 0x7ff82811e1b0 con 0x7ff828105340 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff8100034e0 con 0x7ff828105340 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 608623429 0 0) 0x7ff82811bdd0 con 0x7ff828102580 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff82811e1b0 con 0x7ff828102580 2024-09-10T18:39:42.595 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff8180020a0 con 0x7ff828105340 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff824001550 con 0x7ff828102580 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2460106500 0 0) 0x7ff82811cfb0 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 >> v1:172.21.15.175:6789/0 conn(0x7ff828102580 legacy=0x7ff8280fe130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 >> v1:172.21.15.131:6789/0 conn(0x7ff828105340 legacy=0x7ff8280fe840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff82811f3b0 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82811e3a0 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.561+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82811e7f0 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.562+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7ff820002060 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.562+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff820005e70 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.563+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff814005010 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.563+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7ff82001c310 con 0x7ff828108ed0 2024-09-10T18:39:42.596 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.563+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7ff82008a2f0 con 0x7ff828108ed0 2024-09-10T18:39:42.597 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.568+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7ff820056fe0 con 0x7ff828108ed0 2024-09-10T18:39:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.732+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff8140052c0 con 0x7ff828108ed0 2024-09-10T18:39:42.734 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.732+0000 7ff81d7fa640 1 -- 172.21.15.131:0/1735531885 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 1313722239) 0x7ff820056980 con 0x7ff828108ed0 2024-09-10T18:39:42.734 INFO:teuthology.orchestra.run.smithi131.stdout:107374182437 2024-09-10T18:39:42.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.734+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 >> v1:172.21.15.131:6800/336444945 conn(0x7ff81006feb0 legacy=0x7ff810072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:42.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.734+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 >> v1:172.21.15.131:6790/0 conn(0x7ff828108ed0 legacy=0x7ff82811a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:42.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.734+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 shutdown_connections 2024-09-10T18:39:42.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.734+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 >> 172.21.15.131:0/1735531885 conn(0x7ff8280f8140 msgr2=0x7ff82810bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:42.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.734+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 shutdown_connections 2024-09-10T18:39:42.736 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:42.734+0000 7ff82ec5f640 1 -- 172.21.15.131:0/1735531885 wait complete. 2024-09-10T18:39:43.571 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:39:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:39:43.799 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:43 smithi131 ceph-mon[29369]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:43.799 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1735531885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:39:43.799 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:43 smithi131 ceph-mon[40885]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:43.800 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1735531885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:39:43.800 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:39:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:39:43.412Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002468614s 2024-09-10T18:39:43.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:43 smithi175 ceph-mon[33399]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:43.872 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1735531885' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:39:43.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.875+0000 7fdb11910640 1 -- 172.21.15.131:0/3215879367 >> v1:172.21.15.131:6790/0 conn(0x7fdb0c107dc0 legacy=0x7fdb0c10a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:43.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.875+0000 7fdb11910640 1 -- 172.21.15.131:0/3215879367 shutdown_connections 2024-09-10T18:39:43.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.875+0000 7fdb11910640 1 -- 172.21.15.131:0/3215879367 >> 172.21.15.131:0/3215879367 conn(0x7fdb0c0f7050 msgr2=0x7fdb0c0f9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:43.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.875+0000 7fdb11910640 1 -- 172.21.15.131:0/3215879367 shutdown_connections 2024-09-10T18:39:43.882 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.875+0000 7fdb11910640 1 -- 172.21.15.131:0/3215879367 wait complete. 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb11910640 1 Processor -- start 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb11910640 1 -- start start 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb11910640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdb0c11bef0 con 0x7fdb0c104260 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb11910640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdb0c11d0f0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb11910640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdb0c11e2f0 con 0x7fdb0c1014f0 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb0a7fc640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdb0c104260 0x7fdb0c105990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37342/0 (socket says 172.21.15.131:37342) 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb0affd640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fdb0c1014f0 0x7fdb0c105280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:49108/0 (socket says 172.21.15.131:49108) 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.876+0000 7fdb0a7fc640 1 -- 172.21.15.131:0/23154270 learned_addr learned my addr 172.21.15.131:0/23154270 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 645027225 0 0) 0x7fdb0c11bef0 con 0x7fdb0c104260 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaf40034e0 con 0x7fdb0c104260 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212091384 0 0) 0x7fdb0c11e2f0 con 0x7fdb0c1014f0 2024-09-10T18:39:43.883 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb0c11bef0 con 0x7fdb0c1014f0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536745628 0 0) 0x7fdb0c11d0f0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb0c11e2f0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2890995144 0 0) 0x7fdb0c11bef0 con 0x7fdb0c1014f0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb0c11d0f0 con 0x7fdb0c1014f0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 241521652 0 0) 0x7fdaf40034e0 con 0x7fdb0c104260 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdb0c11bef0 con 0x7fdb0c104260 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1899858829 0 0) 0x7fdb0c11e2f0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdaf40034e0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdafc001830 con 0x7fdb0c1014f0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdb000028f0 con 0x7fdb0c104260 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdb040023e0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.884 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 919506092 0 0) 0x7fdaf40034e0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 >> v1:172.21.15.131:6790/0 conn(0x7fdb0c1014f0 legacy=0x7fdb0c105280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 >> v1:172.21.15.131:6789/0 conn(0x7fdb0c104260 legacy=0x7fdb0c105990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c11f4f0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb0c11c0e0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb0c11c5b0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fdb04003110 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.877+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdb040036a0 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.878+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf8005010 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.879+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fdb04001b80 con 0x7fdb0c107dc0 2024-09-10T18:39:43.885 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.879+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fdb04087960 con 0x7fdb0c107dc0 2024-09-10T18:39:43.886 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:43.883+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdb04054650 con 0x7fdb0c107dc0 2024-09-10T18:39:44.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.046+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 --> v1:172.21.15.175:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fdaf80052c0 con 0x7fdb0c107dc0 2024-09-10T18:39:44.048 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.046+0000 7fdaf37fe640 1 -- 172.21.15.131:0/23154270 <== mon.1 v1:172.21.15.175:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 2990679781) 0x7fdb04001550 con 0x7fdb0c107dc0 2024-09-10T18:39:44.048 INFO:teuthology.orchestra.run.smithi131.stdout:180388626456 2024-09-10T18:39:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.049+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 >> v1:172.21.15.131:6800/336444945 conn(0x7fdaf406fea0 legacy=0x7fdaf4072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.049+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 >> v1:172.21.15.175:6789/0 conn(0x7fdb0c107dc0 legacy=0x7fdb0c11a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.049+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 shutdown_connections 2024-09-10T18:39:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.049+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 >> 172.21.15.131:0/23154270 conn(0x7fdb0c0f7050 msgr2=0x7fdb0c0fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.049+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 shutdown_connections 2024-09-10T18:39:44.051 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.049+0000 7fdb11910640 1 -- 172.21.15.131:0/23154270 wait complete. 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.076+0000 7f243d550640 1 -- 172.21.15.131:0/2970371437 >> v1:172.21.15.131:6789/0 conn(0x7f2438108ef0 legacy=0x7f243810b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.076+0000 7f243d550640 1 -- 172.21.15.131:0/2970371437 shutdown_connections 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.076+0000 7f243d550640 1 -- 172.21.15.131:0/2970371437 >> 172.21.15.131:0/2970371437 conn(0x7f24380f7070 msgr2=0x7f24380f9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.076+0000 7f243d550640 1 -- 172.21.15.131:0/2970371437 shutdown_connections 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.076+0000 7f243d550640 1 -- 172.21.15.131:0/2970371437 wait complete. 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f243d550640 1 Processor -- start 2024-09-10T18:39:44.090 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f243d550640 1 -- start start 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f243d550640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f243811c330 con 0x7f24380fd5c0 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f243d550640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f243811d4f0 con 0x7f2438118580 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f243d550640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f243811e6f0 con 0x7f2438108ef0 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242ffff640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f2438108ef0 0x7f2438104600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:49122/0 (socket says 172.21.15.131:49122) 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242ffff640 1 -- 172.21.15.131:0/867737129 learned_addr learned my addr 172.21.15.131:0/867737129 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f2436ffd640 1 --1- 172.21.15.131:0/867737129 >> v1:172.21.15.131:6789/0 conn(0x7f24380fd5c0 0x7f2438103eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37352/0 (socket says 172.21.15.131:37352) 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020860809 0 0) 0x7f243811e6f0 con 0x7f2438108ef0 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24240034e0 con 0x7f2438108ef0 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768098011 0 0) 0x7f243811c330 con 0x7f24380fd5c0 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f243811e6f0 con 0x7f24380fd5c0 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966402846 0 0) 0x7f243811d4f0 con 0x7f2438118580 2024-09-10T18:39:44.091 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f243811c330 con 0x7f2438118580 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.077+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3319310615 0 0) 0x7f243811e6f0 con 0x7f24380fd5c0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f243811d4f0 con 0x7f24380fd5c0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2946762754 0 0) 0x7f24240034e0 con 0x7f2438108ef0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f243811e6f0 con 0x7f2438108ef0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f2428001950 con 0x7f24380fd5c0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f24200028f0 con 0x7f2438108ef0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1268059265 0 0) 0x7f243811c330 con 0x7f2438118580 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f24240034e0 con 0x7f2438118580 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f2430002d50 con 0x7f2438118580 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 196953340 0 0) 0x7f243811d4f0 con 0x7f24380fd5c0 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 >> v1:172.21.15.131:6790/0 conn(0x7f2438108ef0 legacy=0x7f2438104600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.092 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 >> v1:172.21.15.175:6789/0 conn(0x7f2438118580 legacy=0x7f243811aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243811f8f0 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243811c520 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f243811c970 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f2428001560 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.078+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f24280058e0 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.079+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2418005010 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.079+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f2428005010 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.080+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f2428089d10 con 0x7f24380fd5c0 2024-09-10T18:39:44.093 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.084+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f2428056a00 con 0x7f24380fd5c0 2024-09-10T18:39:44.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.242+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24180052c0 con 0x7f24380fd5c0 2024-09-10T18:39:44.245 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.242+0000 7f242f7fe640 1 -- 172.21.15.131:0/867737129 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3833507532) 0x7f24280563a0 con 0x7f24380fd5c0 2024-09-10T18:39:44.246 INFO:teuthology.orchestra.run.smithi131.stdout:77309411371 2024-09-10T18:39:44.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.244+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 >> v1:172.21.15.131:6800/336444945 conn(0x7f242406fe00 legacy=0x7f24240722b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.244+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 >> v1:172.21.15.131:6789/0 conn(0x7f24380fd5c0 legacy=0x7f2438103eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:44.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.244+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 shutdown_connections 2024-09-10T18:39:44.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.244+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 >> 172.21.15.131:0/867737129 conn(0x7f24380f7070 msgr2=0x7f24380f9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:44.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.244+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 shutdown_connections 2024-09-10T18:39:44.247 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:44.244+0000 7f243d550640 1 -- 172.21.15.131:0/867737129 wait complete. 2024-09-10T18:39:44.753 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:44.753 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/23154270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:39:44.754 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/867737129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:39:44.754 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:44.754 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/23154270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:39:44.754 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/867737129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:39:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/23154270' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:39:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/867737129' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:39:45.225 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182433 got 107374182437 for osd.3 2024-09-10T18:39:45.226 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:45.298 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:39:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:45.298 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 podman[67004]: 2024-09-10 18:39:45.184901502 +0000 UTC m=+0.131258028 container create 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, architecture=x86_64, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.expose-services=, vendor=Red Hat, Inc., distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, io.buildah.version=1.32.0, name=ubi8, com.redhat.component=ubi8-container, release=239.1651231664, version=8.5, description=Ceph Grafana Container) 2024-09-10T18:39:45.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 podman[67004]: 2024-09-10 18:39:45.378678666 +0000 UTC m=+0.325035209 container init 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, version=8.5, name=ubi8, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , release=239.1651231664, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897) 2024-09-10T18:39:45.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 podman[67004]: 2024-09-10 18:39:45.383469843 +0000 UTC m=+0.329826369 container start 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., maintainer=Paul Cuzner , version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.tags=base rhel8, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, io.openshift.expose-services=, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, vcs-type=git) 2024-09-10T18:39:45.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 bash[67004]: 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 2024-09-10T18:39:45.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 systemd[1]: Started Ceph grafana.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:39:45.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536173957Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:39:45.549 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536361422Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536379702Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536387196Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536393173Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536398891Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536404496Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536410247Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.53641607Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536421751Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536427224Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536432537Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536438215Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:39:45.550 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=settings t=2024-09-10T18:39:45.536443834Z level=info msg="App mode production" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=sqlstore t=2024-09-10T18:39:45.53649285Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=sqlstore t=2024-09-10T18:39:45.536509459Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.537329947Z level=info msg="Starting DB migrations" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.537403311Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.537936421Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.53841187Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.538986077Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.539475855Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.540099288Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.540485974Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.541182735Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.541559879Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:39:45.551 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.541916004Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:39:45.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.542330879Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:39:45.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.542651242Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:39:45.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.543017205Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:39:45.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.545352715Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:39:45.552 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.547987965Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:39:45.800 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-mon[33399]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:39:45.800 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:45.800 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:45.800 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:45.800 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.550664235Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.553347006Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.55423335Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.555236079Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.555963035Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.55712011Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.55814618Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.55880809Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:39:45.802 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.559873204Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.560646713Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.56139983Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.56237708Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.56308095Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.564048469Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.564715895Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.565090278Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.565963969Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.566900493Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.567701935Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.568091936Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.569103981Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:39:45.803 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.5698686Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.570808983Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.571578201Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.572464388Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.572868722Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.57326269Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.574185566Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.575166063Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.575899454Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.576844953Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.577638036Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.578473234Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.579422672Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:39:45.804 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.580271073Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.581039253Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.58205586Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.584824809Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.587617476Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.590392897Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.593220182Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.59609153Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.598884693Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.599273297Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.602015157Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.604925425Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:39:45.805 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.607728927Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.608253481Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.610980478Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.611476887Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.613906432Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.614634551Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.616708841Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.617241147Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.620104969Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.622727916Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.625472419Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.625777284Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.628537996Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:39:45.806 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.630974155Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.633558619Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.634090348Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.636920821Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.639428659Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.641973955Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.642297457Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.644990794Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.645416945Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.647691257Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.650285495Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.650922036Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.653571007Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:39:45.807 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.654429483Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:39:45.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.65539363Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:39:45.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.656103105Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:39:45.808 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.657059315Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:39:45.808 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626453 got 180388626456 for osd.6 2024-09-10T18:39:45.808 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:45.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.657817031Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:39:45.809 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.658703007Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.659123608Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.659970847Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.660820776Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.661791315Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.66242682Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.663201587Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.663682755Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.664717502Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.665664672Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.666282642Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.66713326Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:39:45.810 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.667962117Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.668893919Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.669455056Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.670232421Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.670587349Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.671155577Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.672030783Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.672347819Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.67318032Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.67352345Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.67404503Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.674932939Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.675781721Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:39:45.811 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.676732871Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.677471633Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.677934176Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.678783452Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.679636003Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.682563919Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.683623056Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.686369532Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.689168985Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.692087539Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.692509262Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.695361356Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.695894957Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:39:45.812 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.698335491Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.701073163Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.701644226Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.704466812Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.705164654Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.707171819Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.709975648Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.710380281Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.710755349Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.711167493Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.711572809Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.711965181Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.712254297Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:39:45.813 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.712513452Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.713083036Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.713622906Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.713903293Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.71429029Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.714694663Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.714987903Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.715370334Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.715774326Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.716345927Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.716636864Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.717058782Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.717346753Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:39:45.814 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.71761136Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.717991503Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.718384385Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.718636376Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.718892956Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.719217407Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.719489729Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.719878196Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.720162146Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.72071932Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.72105831Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.721667085Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.722249376Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:39:45.815 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.722542938Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.722959305Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.72339679Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.724212322Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.72467498Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.725472749Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.725879648Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.726247848Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.726652284Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.727041696Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.728443996Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.7288179Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:39:45.816 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.729274994Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:39:45.817 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.729566508Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:39:45.817 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.729898335Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:39:45.817 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.7302802Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:39:45.817 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.730861793Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:39:45.817 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.731470982Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:39:45.817 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.732106562Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.732715037Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.733142341Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.733423896Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.733687024Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.734067701Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.734469061Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.734811942Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.735211218Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.735650505Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.736320721Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.736990155Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.737309374Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:39:45.818 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.738323715Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.739075338Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.739962737Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.74026681Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.741090153Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.741552675Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.742050997Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.742720548Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.743142484Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.743604304Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.74440506Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.745279279Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.74614413Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:39:45.819 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.746463802Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.747353078Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.74827675Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.749126677Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.749568717Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.752514016Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.755284935Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.758053412Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.758458433Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.761307852Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.764077028Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.766953222Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.767401411Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:39:45.820 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.770198142Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.773087797Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.775957279Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.77640008Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.779371893Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.780328652Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.782175111Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.78472485Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.785088802Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.787754187Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.790620547Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.79331352Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:39:45.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.793913808Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:39:45.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.796584548Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:39:45.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.799357068Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:39:45.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.801830535Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:39:45.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.802258939Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:39:45.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.802733059Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:39:45.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.805470698Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:39:46.051 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.80809828Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.81071956Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.813460974Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.81615961Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.818832934Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.821440451Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.824151046Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.826763756Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.827550683Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.828385949Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.829238441Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.830087477Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:39:46.052 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.830888464Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.831734621Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.832587819Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.833380054Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.833843793Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.834283404Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.835138542Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.835942193Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.836368494Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.836856548Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.837270838Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.837565669Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.83798964Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:39:46.053 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.838415254Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.838797098Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.839230251Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.839607909Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.841490351Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.841893565Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.842319976Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.842612544Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.842991629Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.843383913Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.843804403Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.844114431Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:39:46.054 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.844820409Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:39:46.055 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.845569455Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.846345224Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.847138694Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.849691367Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.852050201Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.854794159Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.857385684Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.860081975Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.862900309Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.865705068Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.868252615Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.87095432Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:39:46.056 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.873807238Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.876304427Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.877325649Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.878044412Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.879018371Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.879912028Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.88064941Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.881511559Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.88245059Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.883228581Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.884039172Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.885008599Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.885730557Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:39:46.057 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.88659121Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.887422033Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.888277906Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.889163684Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.889975709Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.890961785Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.891707973Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.892689362Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.893464742Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.894356458Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.89492109Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.895334816Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.896194648Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:39:46.058 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.897184446Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.897928681Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.901347456Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.904694257Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.905534342Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.906507525Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.907456897Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.908296165Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.909220948Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.910052614Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.911033755Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.913902709Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.916626298Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:39:46.059 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.919517637Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.922270883Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.925344182Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.928005591Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.930772941Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.933591315Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.936398048Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.93938553Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.942180012Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.944934992Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.947831038Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.950741702Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.953448715Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:39:46.060 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.956222277Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:39:46.061 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.958803467Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.961698052Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.964602669Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.967212076Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.970046261Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.972763197Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.975552856Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.978118902Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.980601343Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:39:46.062 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.983294638Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.986058304Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.988661796Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.991391908Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.994014478Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.996641067Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:45.999340104Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.002054407Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.004535901Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.007117451Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.007964511Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:39:46.063 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.00882153Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.009751507Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.009823376Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.010613794Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.011418942Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.012389988Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.012633777Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.012739317Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.013146221Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.013935398Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:39:46.064 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.014780707Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.015626101Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.016578156Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.017423104Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.018203482Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.019125082Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.019872943Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.020877136Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.025252154Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.026180549Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.027160777Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:39:46.065 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.03154481Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.03579366Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.036866033Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.037502546Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.03842035Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.038929242Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.039359718Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.040209999Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.041186833Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:39:46.066 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.042177367Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:39:46.067 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.043095418Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:39:46.067 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.043952517Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:39:46.067 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.044935821Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:39:46.067 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.045651242Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:39:46.067 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.046532721Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:39:46.067 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.047391234Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:39:46.068 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.050254234Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:39:46.068 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.053036484Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[40885]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[29369]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:45 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:46.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411371 for osd.2 2024-09-10T18:39:46.398 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:46.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.05584971Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:39:46.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.058619678Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:39:46.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.061359968Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.064272608Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.067208295Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.069994912Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.072944264Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.075856966Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.078593298Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.081369675Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.084080946Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.087103305Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.087565408Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:39:46.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.088009058Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.088985647Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.089403718Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.089847131Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.09014821Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.090541153Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.091009967Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.091379579Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.09166779Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.092064288Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.092464025Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.092899965Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:39:46.429 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.093857913Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.094167919Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.094579098Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.095006435Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.095442285Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.096446581Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.096880777Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.097653134Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.09776205Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.098057402Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.098149067Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:39:46.430 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.098427003Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.09922229Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.099744455Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.10020932Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.101058253Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.101877116Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.104831623Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.107698964Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:39:46.431 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.110481596Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.113268762Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.115936199Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.118955984Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.121955393Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.124770384Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.127594466Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.130441415Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.133358848Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.136048656Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.138772348Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.141620268Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:39:46.432 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.142112941Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.144311084Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.147216104Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.149790052Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.152580717Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.155244148Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.158042324Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.160791006Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.163124421Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.165668631Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.168313223Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.171001606Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.173706476Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:39:46.433 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.176441832Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.179203665Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.181984547Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.191921037Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.194676554Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.197390123Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.201006281Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.2037273Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.206469326Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.207352566Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.208009833Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.208808684Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.209655222Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:39:46.434 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=migrator t=2024-09-10T18:39:46.210336535Z level=info msg="migrations completed" performed=485 skipped=0 duration=672.952175ms 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=sqlstore t=2024-09-10T18:39:46.210758288Z level=info msg="Created default organization" 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=plugin.loader t=2024-09-10T18:39:46.266165044Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=plugin.loader t=2024-09-10T18:39:46.293191897Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=plugin.loader t=2024-09-10T18:39:46.29320861Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=secrets t=2024-09-10T18:39:46.293241089Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=query_data t=2024-09-10T18:39:46.294921069Z level=info msg="Query Service initialization" 2024-09-10T18:39:46.435 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=live.push_http t=2024-09-10T18:39:46.297419484Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:39:46.630 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:46 smithi131 ceph-mon[29369]: Deploying daemon node-exporter.a on smithi131 2024-09-10T18:39:46.630 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:46 smithi131 ceph-mon[40885]: Deploying daemon node-exporter.a on smithi131 2024-09-10T18:39:46.803 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:46 smithi175 ceph-mon[33399]: Deploying daemon node-exporter.a on smithi131 2024-09-10T18:39:46.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.853+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/3403155256 >> v1:172.21.15.131:6790/0 conn(0x7f9abc1052e0 legacy=0x7f9abc1076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:46.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.853+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/3403155256 shutdown_connections 2024-09-10T18:39:46.860 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.853+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/3403155256 >> 172.21.15.131:0/3403155256 conn(0x7f9abc074a50 msgr2=0x7f9abc0f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/3403155256 shutdown_connections 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/3403155256 wait complete. 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 Processor -- start 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 -- start start 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9abc11bee0 con 0x7f9abc1025f0 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9abc11d0e0 con 0x7f9abc108f30 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac3ca8640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9abc11e2e0 con 0x7f9abc1052e0 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac1243640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f9abc1052e0 0x7f9abc0feb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:49142/0 (socket says 172.21.15.131:49142) 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac1243640 1 -- 172.21.15.131:0/907923495 learned_addr learned my addr 172.21.15.131:0/907923495 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ac1a44640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f9abc1025f0 0x7f9abc0fe420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37364/0 (socket says 172.21.15.131:37364) 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.854+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526179789 0 0) 0x7f9abc11bee0 con 0x7f9abc1025f0 2024-09-10T18:39:46.861 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa8003120 con 0x7f9abc1025f0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664012279 0 0) 0x7f9abc11e2e0 con 0x7f9abc1052e0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9abc11bee0 con 0x7f9abc1052e0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335377034 0 0) 0x7f9abc11d0e0 con 0x7f9abc108f30 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9abc11e2e0 con 0x7f9abc108f30 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 923476193 0 0) 0x7f9abc11bee0 con 0x7f9abc1052e0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9abc11d0e0 con 0x7f9abc1052e0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9aac0014f0 con 0x7f9abc1052e0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2481878795 0 0) 0x7f9aa8003120 con 0x7f9abc1025f0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9abc11bee0 con 0x7f9abc1025f0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9ab8001a80 con 0x7f9abc1025f0 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1756717570 0 0) 0x7f9abc11e2e0 con 0x7f9abc108f30 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9aa8003120 con 0x7f9abc108f30 2024-09-10T18:39:46.862 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9ab4001c40 con 0x7f9abc108f30 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3411542699 0 0) 0x7f9abc11d0e0 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 >> v1:172.21.15.175:6789/0 conn(0x7f9abc108f30 legacy=0x7f9abc11a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 >> v1:172.21.15.131:6789/0 conn(0x7f9abc1025f0 legacy=0x7f9abc0fe420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc11f4e0 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9abc11e4d0 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9abc11e920 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f9aac0023d0 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.855+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f9aac003710 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.856+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa0005010 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.857+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f9aac001f80 con 0x7f9abc1052e0 2024-09-10T18:39:46.863 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.857+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7f9aac08a120 con 0x7f9abc1052e0 2024-09-10T18:39:46.864 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:46.861+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f9aac056e70 con 0x7f9abc1052e0 2024-09-10T18:39:47.038 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.036+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9aa00052c0 con 0x7f9abc1052e0 2024-09-10T18:39:47.039 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.037+0000 7f9ab27fc640 1 -- 172.21.15.131:0/907923495 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 4206608905) 0x7f9aac056810 con 0x7f9abc1052e0 2024-09-10T18:39:47.039 INFO:teuthology.orchestra.run.smithi131.stdout:206158430228 2024-09-10T18:39:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.039+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 >> v1:172.21.15.131:6800/336444945 conn(0x7f9aa806fda0 legacy=0x7f9aa8072250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.039+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 >> v1:172.21.15.131:6790/0 conn(0x7f9abc1052e0 legacy=0x7f9abc0feb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.039+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 shutdown_connections 2024-09-10T18:39:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.039+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 >> 172.21.15.131:0/907923495 conn(0x7f9abc074a50 msgr2=0x7f9abc10bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.040+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 shutdown_connections 2024-09-10T18:39:47.041 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:47.040+0000 7f9ac3ca8640 1 -- 172.21.15.131:0/907923495 wait complete. 2024-09-10T18:39:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=infra.usagestats.collector t=2024-09-10T18:39:47.074085314Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:39:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=provisioning.datasources t=2024-09-10T18:39:47.074372284Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:39:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=provisioning.datasources t=2024-09-10T18:39:47.10765559Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:39:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=provisioning.datasources t=2024-09-10T18:39:47.139470458Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:39:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=provisioning.alerting t=2024-09-10T18:39:47.16114189Z level=info msg="starting to provision alerting" 2024-09-10T18:39:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=provisioning.alerting t=2024-09-10T18:39:47.161179582Z level=info msg="finished to provision alerting" 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=grafanaStorageLogger t=2024-09-10T18:39:47.16135592Z level=info msg="storage starting" 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=ngalert.state.manager t=2024-09-10T18:39:47.161422606Z level=info msg="Warming state cache for startup" 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=http.server t=2024-09-10T18:39:47.163445353Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=sqlstore.transactions t=2024-09-10T18:39:47.171827999Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=ngalert.state.manager t=2024-09-10T18:39:47.177842286Z level=info msg="State cache has been initialized" states=0 duration=16.414643ms 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=ticker t=2024-09-10T18:39:47.178180235Z level=info msg=starting first_tick=2024-09-10T18:39:50Z 2024-09-10T18:39:47.428 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:39:47.178262506Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:39:47.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[29369]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:47.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/907923495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:39:47.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:47.607 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:47.607 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[40885]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:47.607 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/907923495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:39:47.608 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:47.608 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-mon[33399]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/907923495' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:39:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:39:48.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.070+0000 7fdf54cf1640 1 -- 172.21.15.131:0/1461534761 >> v1:172.21.15.131:6790/0 conn(0x7fdf50104270 legacy=0x7fdf50106650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:48.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- 172.21.15.131:0/1461534761 shutdown_connections 2024-09-10T18:39:48.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- 172.21.15.131:0/1461534761 >> 172.21.15.131:0/1461534761 conn(0x7fdf500f7000 msgr2=0x7fdf500f9410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:48.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- 172.21.15.131:0/1461534761 shutdown_connections 2024-09-10T18:39:48.085 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- 172.21.15.131:0/1461534761 wait complete. 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 Processor -- start 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- start start 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdf50107c10 con 0x7fdf50101500 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdf5011d2b0 con 0x7fdf50107dd0 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf54cf1640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdf5011e4d0 con 0x7fdf50104270 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf4e59c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fdf50101500 0x7fdf50107080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:37382/0 (socket says 172.21.15.131:37382) 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.071+0000 7fdf4e59c640 1 -- 172.21.15.131:0/2591195162 learned_addr learned my addr 172.21.15.131:0/2591195162 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052099551 0 0) 0x7fdf50107c10 con 0x7fdf50101500 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf3c0034e0 con 0x7fdf50101500 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079707784 0 0) 0x7fdf5011e4d0 con 0x7fdf50104270 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf50107c10 con 0x7fdf50104270 2024-09-10T18:39:48.086 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508516102 0 0) 0x7fdf5011d2b0 con 0x7fdf50107dd0 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf5011e4d0 con 0x7fdf50107dd0 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3300607121 0 0) 0x7fdf50107c10 con 0x7fdf50104270 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf5011d2b0 con 0x7fdf50104270 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2236850495 0 0) 0x7fdf3c0034e0 con 0x7fdf50101500 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf50107c10 con 0x7fdf50101500 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdf38001540 con 0x7fdf50104270 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdf40002990 con 0x7fdf50101500 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2278220265 0 0) 0x7fdf5011e4d0 con 0x7fdf50107dd0 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdf3c0034e0 con 0x7fdf50107dd0 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdf48001b00 con 0x7fdf50107dd0 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3030289435 0 0) 0x7fdf5011d2b0 con 0x7fdf50104270 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 >> v1:172.21.15.175:6789/0 conn(0x7fdf50107dd0 legacy=0x7fdf5011ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 >> v1:172.21.15.131:6789/0 conn(0x7fdf50101500 legacy=0x7fdf50107080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:48.087 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5011f6d0 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5011e6c0 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5011eb90 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fdf380017c0 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.072+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fdf38003440 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.073+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf30005010 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.074+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fdf380021a0 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.074+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fdf380877f0 con 0x7fdf50104270 2024-09-10T18:39:48.088 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.078+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fdf380544e0 con 0x7fdf50104270 2024-09-10T18:39:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.241+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fdf300052c0 con 0x7fdf50104270 2024-09-10T18:39:48.244 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.242+0000 7fdf46ffd640 1 -- 172.21.15.131:0/2591195162 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 770812263) 0x7fdf38053e80 con 0x7fdf50104270 2024-09-10T18:39:48.244 INFO:teuthology.orchestra.run.smithi131.stdout:128849018915 2024-09-10T18:39:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.244+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 >> v1:172.21.15.131:6800/336444945 conn(0x7fdf3c06fe80 legacy=0x7fdf3c072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.244+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 >> v1:172.21.15.131:6790/0 conn(0x7fdf50104270 legacy=0x7fdf50117410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.244+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 shutdown_connections 2024-09-10T18:39:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.244+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 >> 172.21.15.131:0/2591195162 conn(0x7fdf500f7000 msgr2=0x7fdf500fb4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.244+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 shutdown_connections 2024-09-10T18:39:48.246 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:48.244+0000 7fdf54cf1640 1 -- 172.21.15.131:0/2591195162 wait complete. 2024-09-10T18:39:48.513 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430225 got 206158430228 for osd.7 2024-09-10T18:39:48.513 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:48.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2591195162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:39:48.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2591195162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:39:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2591195162' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:39:49.186 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018910 got 128849018915 for osd.4 2024-09-10T18:39:49.186 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:49.186 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:39:49.187 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:39:49.458 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:49.577 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:49 smithi131 bash[77178]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:39:49.889 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:49 smithi131 ceph-mon[29369]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:49.889 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:49 smithi131 ceph-mon[40885]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:49.890 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:49 smithi131 bash[77178]: Getting image source signatures 2024-09-10T18:39:49.890 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:49 smithi131 bash[77178]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:39:49.890 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:49 smithi131 bash[77178]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:39:49.890 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:49 smithi131 bash[77178]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:39:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:49 smithi175 ceph-mon[33399]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:51.736 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:51 smithi131 ceph-mon[29369]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:51.736 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:51 smithi131 bash[77178]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:39:51.736 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:51 smithi131 ceph-mon[40885]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:51 smithi175 ceph-mon[33399]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:51.988 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:51 smithi131 bash[77178]: Writing manifest to image destination 2024-09-10T18:39:52.073 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.986+0000 7ff9b940f640 1 -- 172.21.15.131:0/548783866 >> v1:172.21.15.131:6789/0 conn(0x7ff9b4107e00 legacy=0x7ff9b410a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.986+0000 7ff9b940f640 1 -- 172.21.15.131:0/548783866 shutdown_connections 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.986+0000 7ff9b940f640 1 -- 172.21.15.131:0/548783866 >> 172.21.15.131:0/548783866 conn(0x7ff9b40752c0 msgr2=0x7ff9b40f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.986+0000 7ff9b940f640 1 -- 172.21.15.131:0/548783866 shutdown_connections 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.986+0000 7ff9b940f640 1 -- 172.21.15.131:0/548783866 wait complete. 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.987+0000 7ff9b940f640 1 Processor -- start 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.987+0000 7ff9b940f640 1 -- start start 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.987+0000 7ff9b940f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9b41079f0 con 0x7ff9b41042a0 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.987+0000 7ff9b940f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9b4107be0 con 0x7ff9b4107e00 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.987+0000 7ff9b940f640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9b41206a0 con 0x7ff9b4101530 2024-09-10T18:39:52.074 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9b2ffd640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7ff9b4101530 0x7ff9b41071f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:43070/0 (socket says 172.21.15.131:43070) 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9b27fc640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7ff9b41042a0 0x7ff9b4119830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45884/0 (socket says 172.21.15.131:45884) 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9b2ffd640 1 -- 172.21.15.131:0/2375845620 learned_addr learned my addr 172.21.15.131:0/2375845620 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328509299 0 0) 0x7ff9b41079f0 con 0x7ff9b41042a0 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a00034e0 con 0x7ff9b41042a0 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363222758 0 0) 0x7ff9b41206a0 con 0x7ff9b4101530 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b41079f0 con 0x7ff9b4101530 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740655726 0 0) 0x7ff9b4107be0 con 0x7ff9b4107e00 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b41206a0 con 0x7ff9b4107e00 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 180460372 0 0) 0x7ff9a00034e0 con 0x7ff9b41042a0 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9b4107be0 con 0x7ff9b41042a0 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff99c0018c0 con 0x7ff9b41042a0 2024-09-10T18:39:52.075 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2109278027 0 0) 0x7ff9b41079f0 con 0x7ff9b4101530 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9a00034e0 con 0x7ff9b4101530 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff9a4001970 con 0x7ff9b4101530 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3316705015 0 0) 0x7ff9b41206a0 con 0x7ff9b4107e00 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9b41079f0 con 0x7ff9b4107e00 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1244205163 0 0) 0x7ff9b4107be0 con 0x7ff9b41042a0 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 >> v1:172.21.15.131:6790/0 conn(0x7ff9b4101530 legacy=0x7ff9b41071f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 >> v1:172.21.15.175:6789/0 conn(0x7ff9b4107e00 legacy=0x7ff9b411cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4121880 con 0x7ff9b41042a0 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b4120890 con 0x7ff9b41042a0 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b4120ce0 con 0x7ff9b41042a0 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7ff99c002450 con 0x7ff9b41042a0 2024-09-10T18:39:52.076 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.988+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff99c005a80 con 0x7ff9b41042a0 2024-09-10T18:39:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.989+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994005010 con 0x7ff9b41042a0 2024-09-10T18:39:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.990+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7ff99c01bf20 con 0x7ff9b41042a0 2024-09-10T18:39:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.991+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7ff99c089dc0 con 0x7ff9b41042a0 2024-09-10T18:39:52.077 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:51.995+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7ff99c056690 con 0x7ff9b41042a0 2024-09-10T18:39:52.153 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.151+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 --> v1:172.21.15.131:6800/336444945 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff994000b90 con 0x7ff9a006fdd0 2024-09-10T18:39:52.162 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.161+0000 7ff9ab7fe640 1 -- 172.21.15.131:0/2375845620 <== mgr.34496 v1:172.21.15.131:6800/336444945 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384018 (unknown 2965378022 0 3420224264) 0x7ff994000b90 con 0x7ff9a006fdd0 2024-09-10T18:39:52.166 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:39:52.169 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T18:39:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.164+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 >> v1:172.21.15.131:6800/336444945 conn(0x7ff9a006fdd0 legacy=0x7ff9a0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.164+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 >> v1:172.21.15.131:6789/0 conn(0x7ff9b41042a0 legacy=0x7ff9b4119830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.164+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 shutdown_connections 2024-09-10T18:39:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.164+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 >> 172.21.15.131:0/2375845620 conn(0x7ff9b40752c0 msgr2=0x7ff9b40fbd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.164+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 shutdown_connections 2024-09-10T18:39:52.172 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:52.164+0000 7ff9b940f640 1 -- 172.21.15.131:0/2375845620 wait complete. 2024-09-10T18:39:52.577 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:52 smithi131 podman[77178]: 2024-09-10 18:39:52.253014509 +0000 UTC m=+3.016131982 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:39:53.077 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:52 smithi131 podman[77178]: 2024-09-10 18:39:52.611266853 +0000 UTC m=+3.374384298 container create ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:39:53.135 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-10T18:39:50.194701+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":929,"num_read_kb":786,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385392,"kb_used_data":6548,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575824,"statfs":{"total":767960285184,"available":765517643776,"internally_reserved":0,"allocated":6705152,"data_stored":3631680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002602"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704541+0000","last_change":"2024-09-10T18:38:25.926949+0000","last_active":"2024-09-10T18:38:37.704541+0000","last_peered":"2024-09-10T18:38:37.704541+0000","last_clean":"2024-09-10T18:38:37.704541+0000","last_became_active":"2024-09-10T18:38:25.925806+0000","last_became_peered":"2024-09-10T18:38:25.925806+0000","last_unstale":"2024-09-10T18:38:37.704541+0000","last_undegraded":"2024-09-10T18:38:37.704541+0000","last_fullsized":"2024-09-10T18:38:37.704541+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:57.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642685+0000","last_change":"2024-09-10T18:38:27.933810+0000","last_active":"2024-09-10T18:38:37.642685+0000","last_peered":"2024-09-10T18:38:37.642685+0000","last_clean":"2024-09-10T18:38:37.642685+0000","last_became_active":"2024-09-10T18:38:27.933367+0000","last_became_peered":"2024-09-10T18:38:27.933367+0000","last_unstale":"2024-09-10T18:38:37.642685+0000","last_undegraded":"2024-09-10T18:38:37.642685+0000","last_fullsized":"2024-09-10T18:38:37.642685+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:30.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643363+0000","last_change":"2024-09-10T18:38:29.935908+0000","last_active":"2024-09-10T18:38:37.643363+0000","last_peered":"2024-09-10T18:38:37.643363+0000","last_clean":"2024-09-10T18:38:37.643363+0000","last_became_active":"2024-09-10T18:38:29.935600+0000","last_became_peered":"2024-09-10T18:38:29.935600+0000","last_unstale":"2024-09-10T18:38:37.643363+0000","last_undegraded":"2024-09-10T18:38:37.643363+0000","last_fullsized":"2024-09-10T18:38:37.643363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:46.492384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642876+0000","last_change":"2024-09-10T18:38:31.951048+0000","last_active":"2024-09-10T18:38:37.642876+0000","last_peered":"2024-09-10T18:38:37.642876+0000","last_clean":"2024-09-10T18:38:37.642876+0000","last_became_active":"2024-09-10T18:38:31.950702+0000","last_became_peered":"2024-09-10T18:38:31.950702+0000","last_unstale":"2024-09-10T18:38:37.642876+0000","last_undegraded":"2024-09-10T18:38:37.642876+0000","last_fullsized":"2024-09-10T18:38:37.642876+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.911650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642258+0000","last_change":"2024-09-10T18:38:32.252268+0000","last_active":"2024-09-10T18:38:37.642258+0000","last_peered":"2024-09-10T18:38:37.642258+0000","last_clean":"2024-09-10T18:38:37.642258+0000","last_became_active":"2024-09-10T18:38:32.251977+0000","last_became_peered":"2024-09-10T18:38:32.251977+0000","last_unstale":"2024-09-10T18:38:37.642258+0000","last_undegraded":"2024-09-10T18:38:37.642258+0000","last_fullsized":"2024-09-10T18:38:37.642258+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:35.397136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643998+0000","last_change":"2024-09-10T18:38:25.930789+0000","last_active":"2024-09-10T18:38:37.643998+0000","last_peered":"2024-09-10T18:38:37.643998+0000","last_clean":"2024-09-10T18:38:37.643998+0000","last_became_active":"2024-09-10T18:38:25.930521+0000","last_became_peered":"2024-09-10T18:38:25.930521+0000","last_unstale":"2024-09-10T18:38:37.643998+0000","last_undegraded":"2024-09-10T18:38:37.643998+0000","last_fullsized":"2024-09-10T18:38:37.643998+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:33.460802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642436+0000","last_change":"2024-09-10T18:38:27.931540+0000","last_active":"2024-09-10T18:38:37.642436+0000","last_peered":"2024-09-10T18:38:37.642436+0000","last_clean":"2024-09-10T18:38:37.642436+0000","last_became_active":"2024-09-10T18:38:27.930987+0000","last_became_peered":"2024-09-10T18:38:27.930987+0000","last_unstale":"2024-09-10T18:38:37.642436+0000","last_undegraded":"2024-09-10T18:38:37.642436+0000","last_fullsized":"2024-09-10T18:38:37.642436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:29.064086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642677+0000","last_change":"2024-09-10T18:38:29.942627+0000","last_active":"2024-09-10T18:38:37.642677+0000","last_peered":"2024-09-10T18:38:37.642677+0000","last_clean":"2024-09-10T18:38:37.642677+0000","last_became_active":"2024-09-10T18:38:29.942358+0000","last_became_peered":"2024-09-10T18:38:29.942358+0000","last_unstale":"2024-09-10T18:38:37.642677+0000","last_undegraded":"2024-09-10T18:38:37.642677+0000","last_fullsized":"2024-09-10T18:38:37.642677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:39.988568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158715+0000","last_change":"2024-09-10T18:38:25.924451+0000","last_active":"2024-09-10T18:38:38.158715+0000","last_peered":"2024-09-10T18:38:38.158715+0000","last_clean":"2024-09-10T18:38:38.158715+0000","last_became_active":"2024-09-10T18:38:25.924203+0000","last_became_peered":"2024-09-10T18:38:25.924203+0000","last_unstale":"2024-09-10T18:38:38.158715+0000","last_undegraded":"2024-09-10T18:38:38.158715+0000","last_fullsized":"2024-09-10T18:38:38.158715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:39.727352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643163+0000","last_change":"2024-09-10T18:38:27.928758+0000","last_active":"2024-09-10T18:38:37.643163+0000","last_peered":"2024-09-10T18:38:37.643163+0000","last_clean":"2024-09-10T18:38:37.643163+0000","last_became_active":"2024-09-10T18:38:27.928561+0000","last_became_peered":"2024-09-10T18:38:27.928561+0000","last_unstale":"2024-09-10T18:38:37.643163+0000","last_undegraded":"2024-09-10T18:38:37.643163+0000","last_fullsized":"2024-09-10T18:38:37.643163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:15.983852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157733+0000","last_change":"2024-09-10T18:38:29.942125+0000","last_active":"2024-09-10T18:38:38.157733+0000","last_peered":"2024-09-10T18:38:38.157733+0000","last_clean":"2024-09-10T18:38:38.157733+0000","last_became_active":"2024-09-10T18:38:29.941829+0000","last_became_peered":"2024-09-10T18:38:29.941829+0000","last_unstale":"2024-09-10T18:38:38.157733+0000","last_undegraded":"2024-09-10T18:38:38.157733+0000","last_fullsized":"2024-09-10T18:38:38.157733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:39.792363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.703948+0000","last_change":"2024-09-10T18:38:31.953741+0000","last_active":"2024-09-10T18:38:37.703948+0000","last_peered":"2024-09-10T18:38:37.703948+0000","last_clean":"2024-09-10T18:38:37.703948+0000","last_became_active":"2024-09-10T18:38:31.952446+0000","last_became_peered":"2024-09-10T18:38:31.952446+0000","last_unstale":"2024-09-10T18:38:37.703948+0000","last_undegraded":"2024-09-10T18:38:37.703948+0000","last_fullsized":"2024-09-10T18:38:37.703948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:55.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158156+0000","last_change":"2024-09-10T18:38:25.926950+0000","last_active":"2024-09-10T18:38:38.158156+0000","last_peered":"2024-09-10T18:38:38.158156+0000","last_clean":"2024-09-10T18:38:38.158156+0000","last_became_active":"2024-09-10T18:38:25.926653+0000","last_became_peered":"2024-09-10T18:38:25.926653+0000","last_unstale":"2024-09-10T18:38:38.158156+0000","last_undegraded":"2024-09-10T18:38:38.158156+0000","last_fullsized":"2024-09-10T18:38:38.158156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:56.063659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642876+0000","last_change":"2024-09-10T18:38:27.933795+0000","last_active":"2024-09-10T18:38:37.642876+0000","last_peered":"2024-09-10T18:38:37.642876+0000","last_clean":"2024-09-10T18:38:37.642876+0000","last_became_active":"2024-09-10T18:38:27.933358+0000","last_became_peered":"2024-09-10T18:38:27.933358+0000","last_unstale":"2024-09-10T18:38:37.642876+0000","last_undegraded":"2024-09-10T18:38:37.642876+0000","last_fullsized":"2024-09-10T18:38:37.642876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:50.231995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702017+0000","last_change":"2024-09-10T18:38:29.938986+0000","last_active":"2024-09-10T18:38:37.702017+0000","last_peered":"2024-09-10T18:38:37.702017+0000","last_clean":"2024-09-10T18:38:37.702017+0000","last_became_active":"2024-09-10T18:38:29.938656+0000","last_became_peered":"2024-09-10T18:38:29.938656+0000","last_unstale":"2024-09-10T18:38:37.702017+0000","last_undegraded":"2024-09-10T18:38:37.702017+0000","last_fullsized":"2024-09-10T18:38:37.702017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:59.184333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156974+0000","last_change":"2024-09-10T18:38:31.950912+0000","last_active":"2024-09-10T18:38:38.156974+0000","last_peered":"2024-09-10T18:38:38.156974+0000","last_clean":"2024-09-10T18:38:38.156974+0000","last_became_active":"2024-09-10T18:38:31.950631+0000","last_became_peered":"2024-09-10T18:38:31.950631+0000","last_unstale":"2024-09-10T18:38:38.156974+0000","last_undegraded":"2024-09-10T18:38:38.156974+0000","last_fullsized":"2024-09-10T18:38:38.156974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:33.589815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644018+0000","last_change":"2024-09-10T18:38:32.250423+0000","last_active":"2024-09-10T18:38:37.644018+0000","last_peered":"2024-09-10T18:38:37.644018+0000","last_clean":"2024-09-10T18:38:37.644018+0000","last_became_active":"2024-09-10T18:38:32.250138+0000","last_became_peered":"2024-09-10T18:38:32.250138+0000","last_unstale":"2024-09-10T18:38:37.644018+0000","last_undegraded":"2024-09-10T18:38:37.644018+0000","last_fullsized":"2024-09-10T18:38:37.644018+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:04.859620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704511+0000","last_change":"2024-09-10T18:38:25.924952+0000","last_active":"2024-09-10T18:38:37.704511+0000","last_peered":"2024-09-10T18:38:37.704511+0000","last_clean":"2024-09-10T18:38:37.704511+0000","last_became_active":"2024-09-10T18:38:25.924666+0000","last_became_peered":"2024-09-10T18:38:25.924666+0000","last_unstale":"2024-09-10T18:38:37.704511+0000","last_undegraded":"2024-09-10T18:38:37.704511+0000","last_fullsized":"2024-09-10T18:38:37.704511+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:01.282908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157054+0000","last_change":"2024-09-10T18:38:27.930615+0000","last_active":"2024-09-10T18:38:38.157054+0000","last_peered":"2024-09-10T18:38:38.157054+0000","last_clean":"2024-09-10T18:38:38.157054+0000","last_became_active":"2024-09-10T18:38:27.930326+0000","last_became_peered":"2024-09-10T18:38:27.930326+0000","last_unstale":"2024-09-10T18:38:38.157054+0000","last_undegraded":"2024-09-10T18:38:38.157054+0000","last_fullsized":"2024-09-10T18:38:38.157054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:55.729441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643648+0000","last_change":"2024-09-10T18:38:29.935741+0000","last_active":"2024-09-10T18:38:37.643648+0000","last_peered":"2024-09-10T18:38:37.643648+0000","last_clean":"2024-09-10T18:38:37.643648+0000","last_became_active":"2024-09-10T18:38:29.935034+0000","last_became_peered":"2024-09-10T18:38:29.935034+0000","last_unstale":"2024-09-10T18:38:37.643648+0000","last_undegraded":"2024-09-10T18:38:37.643648+0000","last_fullsized":"2024-09-10T18:38:37.643648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:06.651867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643300+0000","last_change":"2024-09-10T18:38:32.249832+0000","last_active":"2024-09-10T18:38:37.643300+0000","last_peered":"2024-09-10T18:38:37.643300+0000","last_clean":"2024-09-10T18:38:37.643300+0000","last_became_active":"2024-09-10T18:38:32.249492+0000","last_became_peered":"2024-09-10T18:38:32.249492+0000","last_unstale":"2024-09-10T18:38:37.643300+0000","last_undegraded":"2024-09-10T18:38:37.643300+0000","last_fullsized":"2024-09-10T18:38:37.643300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:39.245093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643231+0000","last_change":"2024-09-10T18:38:25.931692+0000","last_active":"2024-09-10T18:38:37.643231+0000","last_peered":"2024-09-10T18:38:37.643231+0000","last_clean":"2024-09-10T18:38:37.643231+0000","last_became_active":"2024-09-10T18:38:25.930675+0000","last_became_peered":"2024-09-10T18:38:25.930675+0000","last_unstale":"2024-09-10T18:38:37.643231+0000","last_undegraded":"2024-09-10T18:38:37.643231+0000","last_fullsized":"2024-09-10T18:38:37.643231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:13.164100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643008+0000","last_change":"2024-09-10T18:38:27.937292+0000","last_active":"2024-09-10T18:38:37.643008+0000","last_peered":"2024-09-10T18:38:37.643008+0000","last_clean":"2024-09-10T18:38:37.643008+0000","last_became_active":"2024-09-10T18:38:27.936980+0000","last_became_peered":"2024-09-10T18:38:27.936980+0000","last_unstale":"2024-09-10T18:38:37.643008+0000","last_undegraded":"2024-09-10T18:38:37.643008+0000","last_fullsized":"2024-09-10T18:38:37.643008+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:35.116701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643328+0000","last_change":"2024-09-10T18:38:29.940856+0000","last_active":"2024-09-10T18:38:37.643328+0000","last_peered":"2024-09-10T18:38:37.643328+0000","last_clean":"2024-09-10T18:38:37.643328+0000","last_became_active":"2024-09-10T18:38:29.940581+0000","last_became_peered":"2024-09-10T18:38:29.940581+0000","last_unstale":"2024-09-10T18:38:37.643328+0000","last_undegraded":"2024-09-10T18:38:37.643328+0000","last_fullsized":"2024-09-10T18:38:37.643328+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.764961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702029+0000","last_change":"2024-09-10T18:38:31.950648+0000","last_active":"2024-09-10T18:38:37.702029+0000","last_peered":"2024-09-10T18:38:37.702029+0000","last_clean":"2024-09-10T18:38:37.702029+0000","last_became_active":"2024-09-10T18:38:31.950370+0000","last_became_peered":"2024-09-10T18:38:31.950370+0000","last_unstale":"2024-09-10T18:38:37.702029+0000","last_undegraded":"2024-09-10T18:38:37.702029+0000","last_fullsized":"2024-09-10T18:38:37.702029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:07.922850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642995+0000","last_change":"2024-09-10T18:38:25.924183+0000","last_active":"2024-09-10T18:38:37.642995+0000","last_peered":"2024-09-10T18:38:37.642995+0000","last_clean":"2024-09-10T18:38:37.642995+0000","last_became_active":"2024-09-10T18:38:25.923881+0000","last_became_peered":"2024-09-10T18:38:25.923881+0000","last_unstale":"2024-09-10T18:38:37.642995+0000","last_undegraded":"2024-09-10T18:38:37.642995+0000","last_fullsized":"2024-09-10T18:38:37.642995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:17.583082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704407+0000","last_change":"2024-09-10T18:38:28.150726+0000","last_active":"2024-09-10T18:38:37.704407+0000","last_peered":"2024-09-10T18:38:37.704407+0000","last_clean":"2024-09-10T18:38:37.704407+0000","last_became_active":"2024-09-10T18:38:28.150471+0000","last_became_peered":"2024-09-10T18:38:28.150471+0000","last_unstale":"2024-09-10T18:38:37.704407+0000","last_undegraded":"2024-09-10T18:38:37.704407+0000","last_fullsized":"2024-09-10T18:38:37.704407+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:44.263683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256569+0000","last_change":"2024-09-10T18:38:29.940452+0000","last_active":"2024-09-10T18:39:39.256569+0000","last_peered":"2024-09-10T18:39:39.256569+0000","last_clean":"2024-09-10T18:39:39.256569+0000","last_became_active":"2024-09-10T18:38:29.939059+0000","last_became_peered":"2024-09-10T18:38:29.939059+0000","last_unstale":"2024-09-10T18:39:39.256569+0000","last_undegraded":"2024-09-10T18:39:39.256569+0000","last_fullsized":"2024-09-10T18:39:39.256569+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644138+0000","last_change":"2024-09-10T18:38:27.935949+0000","last_active":"2024-09-10T18:38:37.644138+0000","last_peered":"2024-09-10T18:38:37.644138+0000","last_clean":"2024-09-10T18:38:37.644138+0000","last_became_active":"2024-09-10T18:38:27.935674+0000","last_became_peered":"2024-09-10T18:38:27.935674+0000","last_unstale":"2024-09-10T18:38:37.644138+0000","last_undegraded":"2024-09-10T18:38:37.644138+0000","last_fullsized":"2024-09-10T18:38:37.644138+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:17.374429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642044+0000","last_change":"2024-09-10T18:38:25.929533+0000","last_active":"2024-09-10T18:38:37.642044+0000","last_peered":"2024-09-10T18:38:37.642044+0000","last_clean":"2024-09-10T18:38:37.642044+0000","last_became_active":"2024-09-10T18:38:25.929214+0000","last_became_peered":"2024-09-10T18:38:25.929214+0000","last_unstale":"2024-09-10T18:38:37.642044+0000","last_undegraded":"2024-09-10T18:38:37.642044+0000","last_fullsized":"2024-09-10T18:38:37.642044+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:43.870184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256554+0000","last_change":"2024-09-10T18:38:29.943705+0000","last_active":"2024-09-10T18:39:39.256554+0000","last_peered":"2024-09-10T18:39:39.256554+0000","last_clean":"2024-09-10T18:39:39.256554+0000","last_became_active":"2024-09-10T18:38:29.943424+0000","last_became_peered":"2024-09-10T18:38:29.943424+0000","last_unstale":"2024-09-10T18:39:39.256554+0000","last_undegraded":"2024-09-10T18:39:39.256554+0000","last_fullsized":"2024-09-10T18:39:39.256554+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:12:36.965628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158152+0000","last_change":"2024-09-10T18:38:31.950934+0000","last_active":"2024-09-10T18:38:38.158152+0000","last_peered":"2024-09-10T18:38:38.158152+0000","last_clean":"2024-09-10T18:38:38.158152+0000","last_became_active":"2024-09-10T18:38:31.950664+0000","last_became_peered":"2024-09-10T18:38:31.950664+0000","last_unstale":"2024-09-10T18:38:38.158152+0000","last_undegraded":"2024-09-10T18:38:38.158152+0000","last_fullsized":"2024-09-10T18:38:38.158152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:02.893403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.641936+0000","last_change":"2024-09-10T18:38:28.150754+0000","last_active":"2024-09-10T18:38:37.641936+0000","last_peered":"2024-09-10T18:38:37.641936+0000","last_clean":"2024-09-10T18:38:37.641936+0000","last_became_active":"2024-09-10T18:38:28.150484+0000","last_became_peered":"2024-09-10T18:38:28.150484+0000","last_unstale":"2024-09-10T18:38:37.641936+0000","last_undegraded":"2024-09-10T18:38:37.641936+0000","last_fullsized":"2024-09-10T18:38:37.641936+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:46.312788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643614+0000","last_change":"2024-09-10T18:38:25.926310+0000","last_active":"2024-09-10T18:38:37.643614+0000","last_peered":"2024-09-10T18:38:37.643614+0000","last_clean":"2024-09-10T18:38:37.643614+0000","last_became_active":"2024-09-10T18:38:25.925756+0000","last_became_peered":"2024-09-10T18:38:25.925756+0000","last_unstale":"2024-09-10T18:38:37.643614+0000","last_undegraded":"2024-09-10T18:38:37.643614+0000","last_fullsized":"2024-09-10T18:38:37.643614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:35.132143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642707+0000","last_change":"2024-09-10T18:38:29.942601+0000","last_active":"2024-09-10T18:38:37.642707+0000","last_peered":"2024-09-10T18:38:37.642707+0000","last_clean":"2024-09-10T18:38:37.642707+0000","last_became_active":"2024-09-10T18:38:29.942125+0000","last_became_peered":"2024-09-10T18:38:29.942125+0000","last_unstale":"2024-09-10T18:38:37.642707+0000","last_undegraded":"2024-09-10T18:38:37.642707+0000","last_fullsized":"2024-09-10T18:38:37.642707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:37.094077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643091+0000","last_change":"2024-09-10T18:38:31.951970+0000","last_active":"2024-09-10T18:38:37.643091+0000","last_peered":"2024-09-10T18:38:37.643091+0000","last_clean":"2024-09-10T18:38:37.643091+0000","last_became_active":"2024-09-10T18:38:31.951643+0000","last_became_peered":"2024-09-10T18:38:31.951643+0000","last_unstale":"2024-09-10T18:38:37.643091+0000","last_undegraded":"2024-09-10T18:38:37.643091+0000","last_fullsized":"2024-09-10T18:38:37.643091+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:35.211919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643246+0000","last_change":"2024-09-10T18:38:27.925972+0000","last_active":"2024-09-10T18:38:37.643246+0000","last_peered":"2024-09-10T18:38:37.643246+0000","last_clean":"2024-09-10T18:38:37.643246+0000","last_became_active":"2024-09-10T18:38:27.925684+0000","last_became_peered":"2024-09-10T18:38:27.925684+0000","last_unstale":"2024-09-10T18:38:37.643246+0000","last_undegraded":"2024-09-10T18:38:37.643246+0000","last_fullsized":"2024-09-10T18:38:37.643246+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:41.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704523+0000","last_change":"2024-09-10T18:38:25.927668+0000","last_active":"2024-09-10T18:38:37.704523+0000","last_peered":"2024-09-10T18:38:37.704523+0000","last_clean":"2024-09-10T18:38:37.704523+0000","last_became_active":"2024-09-10T18:38:25.927255+0000","last_became_peered":"2024-09-10T18:38:25.927255+0000","last_unstale":"2024-09-10T18:38:37.704523+0000","last_undegraded":"2024-09-10T18:38:37.704523+0000","last_fullsized":"2024-09-10T18:38:37.704523+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:59.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256638+0000","last_change":"2024-09-10T18:38:29.938812+0000","last_active":"2024-09-10T18:39:39.256638+0000","last_peered":"2024-09-10T18:39:39.256638+0000","last_clean":"2024-09-10T18:39:39.256638+0000","last_became_active":"2024-09-10T18:38:29.938529+0000","last_became_peered":"2024-09-10T18:38:29.938529+0000","last_unstale":"2024-09-10T18:39:39.256638+0000","last_undegraded":"2024-09-10T18:39:39.256638+0000","last_fullsized":"2024-09-10T18:39:39.256638+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:22.174174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702102+0000","last_change":"2024-09-10T18:38:32.250387+0000","last_active":"2024-09-10T18:38:37.702102+0000","last_peered":"2024-09-10T18:38:37.702102+0000","last_clean":"2024-09-10T18:38:37.702102+0000","last_became_active":"2024-09-10T18:38:32.250109+0000","last_became_peered":"2024-09-10T18:38:32.250109+0000","last_unstale":"2024-09-10T18:38:37.702102+0000","last_undegraded":"2024-09-10T18:38:37.702102+0000","last_fullsized":"2024-09-10T18:38:37.702102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:42.853793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642782+0000","last_change":"2024-09-10T18:38:27.932642+0000","last_active":"2024-09-10T18:38:37.642782+0000","last_peered":"2024-09-10T18:38:37.642782+0000","last_clean":"2024-09-10T18:38:37.642782+0000","last_became_active":"2024-09-10T18:38:27.932162+0000","last_became_peered":"2024-09-10T18:38:27.932162+0000","last_unstale":"2024-09-10T18:38:37.642782+0000","last_undegraded":"2024-09-10T18:38:37.642782+0000","last_fullsized":"2024-09-10T18:38:37.642782+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:59.099416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158724+0000","last_change":"2024-09-10T18:38:25.929666+0000","last_active":"2024-09-10T18:38:38.158724+0000","last_peered":"2024-09-10T18:38:38.158724+0000","last_clean":"2024-09-10T18:38:38.158724+0000","last_became_active":"2024-09-10T18:38:25.929395+0000","last_became_peered":"2024-09-10T18:38:25.929395+0000","last_unstale":"2024-09-10T18:38:38.158724+0000","last_undegraded":"2024-09-10T18:38:38.158724+0000","last_fullsized":"2024-09-10T18:38:38.158724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:46.085963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":41,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256606+0000","last_change":"2024-09-10T18:38:29.939121+0000","last_active":"2024-09-10T18:39:39.256606+0000","last_peered":"2024-09-10T18:39:39.256606+0000","last_clean":"2024-09-10T18:39:39.256606+0000","last_became_active":"2024-09-10T18:38:29.938836+0000","last_became_peered":"2024-09-10T18:38:29.938836+0000","last_unstale":"2024-09-10T18:39:39.256606+0000","last_undegraded":"2024-09-10T18:39:39.256606+0000","last_fullsized":"2024-09-10T18:39:39.256606+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:03.303855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704074+0000","last_change":"2024-09-10T18:38:31.953554+0000","last_active":"2024-09-10T18:38:37.704074+0000","last_peered":"2024-09-10T18:38:37.704074+0000","last_clean":"2024-09-10T18:38:37.704074+0000","last_became_active":"2024-09-10T18:38:31.951868+0000","last_became_peered":"2024-09-10T18:38:31.951868+0000","last_unstale":"2024-09-10T18:38:37.704074+0000","last_undegraded":"2024-09-10T18:38:37.704074+0000","last_fullsized":"2024-09-10T18:38:37.704074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:02.862015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704339+0000","last_change":"2024-09-10T18:38:28.149464+0000","last_active":"2024-09-10T18:38:37.704339+0000","last_peered":"2024-09-10T18:38:37.704339+0000","last_clean":"2024-09-10T18:38:37.704339+0000","last_became_active":"2024-09-10T18:38:28.149177+0000","last_became_peered":"2024-09-10T18:38:28.149177+0000","last_unstale":"2024-09-10T18:38:37.704339+0000","last_undegraded":"2024-09-10T18:38:37.704339+0000","last_fullsized":"2024-09-10T18:38:37.704339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:28.427327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644484+0000","last_change":"2024-09-10T18:38:25.927418+0000","last_active":"2024-09-10T18:38:37.644484+0000","last_peered":"2024-09-10T18:38:37.644484+0000","last_clean":"2024-09-10T18:38:37.644484+0000","last_became_active":"2024-09-10T18:38:25.927115+0000","last_became_peered":"2024-09-10T18:38:25.927115+0000","last_unstale":"2024-09-10T18:38:37.644484+0000","last_undegraded":"2024-09-10T18:38:37.644484+0000","last_fullsized":"2024-09-10T18:38:37.644484+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:29.778298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156929+0000","last_change":"2024-09-10T18:38:29.937895+0000","last_active":"2024-09-10T18:38:38.156929+0000","last_peered":"2024-09-10T18:38:38.156929+0000","last_clean":"2024-09-10T18:38:38.156929+0000","last_became_active":"2024-09-10T18:38:29.937631+0000","last_became_peered":"2024-09-10T18:38:29.937631+0000","last_unstale":"2024-09-10T18:38:38.156929+0000","last_undegraded":"2024-09-10T18:38:38.156929+0000","last_fullsized":"2024-09-10T18:38:38.156929+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.858281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642633+0000","last_change":"2024-09-10T18:38:32.250040+0000","last_active":"2024-09-10T18:38:37.642633+0000","last_peered":"2024-09-10T18:38:37.642633+0000","last_clean":"2024-09-10T18:38:37.642633+0000","last_became_active":"2024-09-10T18:38:32.249721+0000","last_became_peered":"2024-09-10T18:38:32.249721+0000","last_unstale":"2024-09-10T18:38:37.642633+0000","last_undegraded":"2024-09-10T18:38:37.642633+0000","last_fullsized":"2024-09-10T18:38:37.642633+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:28.491717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256655+0000","last_change":"2024-09-10T18:38:27.933681+0000","last_active":"2024-09-10T18:39:39.256655+0000","last_peered":"2024-09-10T18:39:39.256655+0000","last_clean":"2024-09-10T18:39:39.256655+0000","last_became_active":"2024-09-10T18:38:27.933424+0000","last_became_peered":"2024-09-10T18:38:27.933424+0000","last_unstale":"2024-09-10T18:39:39.256655+0000","last_undegraded":"2024-09-10T18:39:39.256655+0000","last_fullsized":"2024-09-10T18:39:39.256655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:13.041194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643205+0000","last_change":"2024-09-10T18:38:25.922397+0000","last_active":"2024-09-10T18:38:37.643205+0000","last_peered":"2024-09-10T18:38:37.643205+0000","last_clean":"2024-09-10T18:38:37.643205+0000","last_became_active":"2024-09-10T18:38:25.922119+0000","last_became_peered":"2024-09-10T18:38:25.922119+0000","last_unstale":"2024-09-10T18:38:37.643205+0000","last_undegraded":"2024-09-10T18:38:37.643205+0000","last_fullsized":"2024-09-10T18:38:37.643205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:09.092791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157586+0000","last_change":"2024-09-10T18:38:29.937535+0000","last_active":"2024-09-10T18:38:38.157586+0000","last_peered":"2024-09-10T18:38:38.157586+0000","last_clean":"2024-09-10T18:38:38.157586+0000","last_became_active":"2024-09-10T18:38:25.926644+0000","last_became_peered":"2024-09-10T18:38:25.926644+0000","last_unstale":"2024-09-10T18:38:38.157586+0000","last_undegraded":"2024-09-10T18:38:38.157586+0000","last_fullsized":"2024-09-10T18:38:38.157586+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:55.764411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027062990000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704332+0000","last_change":"2024-09-10T18:38:29.934156+0000","last_active":"2024-09-10T18:38:37.704332+0000","last_peered":"2024-09-10T18:38:37.704332+0000","last_clean":"2024-09-10T18:38:37.704332+0000","last_became_active":"2024-09-10T18:38:29.933852+0000","last_became_peered":"2024-09-10T18:38:29.933852+0000","last_unstale":"2024-09-10T18:38:37.704332+0000","last_undegraded":"2024-09-10T18:38:37.704332+0000","last_fullsized":"2024-09-10T18:38:37.704332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:26.921552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643807+0000","last_change":"2024-09-10T18:38:31.951149+0000","last_active":"2024-09-10T18:38:37.643807+0000","last_peered":"2024-09-10T18:38:37.643807+0000","last_clean":"2024-09-10T18:38:37.643807+0000","last_became_active":"2024-09-10T18:38:31.950873+0000","last_became_peered":"2024-09-10T18:38:31.950873+0000","last_unstale":"2024-09-10T18:38:37.643807+0000","last_undegraded":"2024-09-10T18:38:37.643807+0000","last_fullsized":"2024-09-10T18:38:37.643807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:54.867493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644202+0000","last_change":"2024-09-10T18:38:27.929798+0000","last_active":"2024-09-10T18:38:37.644202+0000","last_peered":"2024-09-10T18:38:37.644202+0000","last_clean":"2024-09-10T18:38:37.644202+0000","last_became_active":"2024-09-10T18:38:27.929406+0000","last_became_peered":"2024-09-10T18:38:27.929406+0000","last_unstale":"2024-09-10T18:38:37.644202+0000","last_undegraded":"2024-09-10T18:38:37.644202+0000","last_fullsized":"2024-09-10T18:38:37.644202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:33.690844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643043+0000","last_change":"2024-09-10T18:38:25.927708+0000","last_active":"2024-09-10T18:38:37.643043+0000","last_peered":"2024-09-10T18:38:37.643043+0000","last_clean":"2024-09-10T18:38:37.643043+0000","last_became_active":"2024-09-10T18:38:25.927326+0000","last_became_peered":"2024-09-10T18:38:25.927326+0000","last_unstale":"2024-09-10T18:38:37.643043+0000","last_undegraded":"2024-09-10T18:38:37.643043+0000","last_fullsized":"2024-09-10T18:38:37.643043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:43.064328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156603+0000","last_change":"2024-09-10T18:38:29.929189+0000","last_active":"2024-09-10T18:38:38.156603+0000","last_peered":"2024-09-10T18:38:38.156603+0000","last_clean":"2024-09-10T18:38:38.156603+0000","last_became_active":"2024-09-10T18:38:25.930264+0000","last_became_peered":"2024-09-10T18:38:25.930264+0000","last_unstale":"2024-09-10T18:38:38.156603+0000","last_undegraded":"2024-09-10T18:38:38.156603+0000","last_fullsized":"2024-09-10T18:38:38.156603+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:54.242883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048772099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156241+0000","last_change":"2024-09-10T18:38:29.942328+0000","last_active":"2024-09-10T18:38:38.156241+0000","last_peered":"2024-09-10T18:38:38.156241+0000","last_clean":"2024-09-10T18:38:38.156241+0000","last_became_active":"2024-09-10T18:38:29.942081+0000","last_became_peered":"2024-09-10T18:38:29.942081+0000","last_unstale":"2024-09-10T18:38:38.156241+0000","last_undegraded":"2024-09-10T18:38:38.156241+0000","last_fullsized":"2024-09-10T18:38:38.156241+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:22.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702344+0000","last_change":"2024-09-10T18:38:32.250727+0000","last_active":"2024-09-10T18:38:37.702344+0000","last_peered":"2024-09-10T18:38:37.702344+0000","last_clean":"2024-09-10T18:38:37.702344+0000","last_became_active":"2024-09-10T18:38:32.250462+0000","last_became_peered":"2024-09-10T18:38:32.250462+0000","last_unstale":"2024-09-10T18:38:37.702344+0000","last_undegraded":"2024-09-10T18:38:37.702344+0000","last_fullsized":"2024-09-10T18:38:37.702344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.779636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704427+0000","last_change":"2024-09-10T18:38:27.931445+0000","last_active":"2024-09-10T18:38:37.704427+0000","last_peered":"2024-09-10T18:38:37.704427+0000","last_clean":"2024-09-10T18:38:37.704427+0000","last_became_active":"2024-09-10T18:38:27.931185+0000","last_became_peered":"2024-09-10T18:38:27.931185+0000","last_unstale":"2024-09-10T18:38:37.704427+0000","last_undegraded":"2024-09-10T18:38:37.704427+0000","last_fullsized":"2024-09-10T18:38:37.704427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:08.329500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704175+0000","last_change":"2024-09-10T18:38:25.924336+0000","last_active":"2024-09-10T18:38:37.704175+0000","last_peered":"2024-09-10T18:38:37.704175+0000","last_clean":"2024-09-10T18:38:37.704175+0000","last_became_active":"2024-09-10T18:38:25.924073+0000","last_became_peered":"2024-09-10T18:38:25.924073+0000","last_unstale":"2024-09-10T18:38:37.704175+0000","last_undegraded":"2024-09-10T18:38:37.704175+0000","last_fullsized":"2024-09-10T18:38:37.704175+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:24.459110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":102,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:46.822802+0000","last_change":"2024-09-10T18:38:29.927353+0000","last_active":"2024-09-10T18:39:46.822802+0000","last_peered":"2024-09-10T18:39:46.822802+0000","last_clean":"2024-09-10T18:39:46.822802+0000","last_became_active":"2024-09-10T18:38:25.928223+0000","last_became_peered":"2024-09-10T18:38:25.928223+0000","last_unstale":"2024-09-10T18:39:46.822802+0000","last_undegraded":"2024-09-10T18:39:46.822802+0000","last_fullsized":"2024-09-10T18:39:46.822802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:51.337256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039067000000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158322+0000","last_change":"2024-09-10T18:38:29.941619+0000","last_active":"2024-09-10T18:38:38.158322+0000","last_peered":"2024-09-10T18:38:38.158322+0000","last_clean":"2024-09-10T18:38:38.158322+0000","last_became_active":"2024-09-10T18:38:29.941242+0000","last_became_peered":"2024-09-10T18:38:29.941242+0000","last_unstale":"2024-09-10T18:38:38.158322+0000","last_undegraded":"2024-09-10T18:38:38.158322+0000","last_fullsized":"2024-09-10T18:38:38.158322+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:09.101368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644083+0000","last_change":"2024-09-10T18:38:31.951193+0000","last_active":"2024-09-10T18:38:37.644083+0000","last_peered":"2024-09-10T18:38:37.644083+0000","last_clean":"2024-09-10T18:38:37.644083+0000","last_became_active":"2024-09-10T18:38:31.950872+0000","last_became_peered":"2024-09-10T18:38:31.950872+0000","last_unstale":"2024-09-10T18:38:37.644083+0000","last_undegraded":"2024-09-10T18:38:37.644083+0000","last_fullsized":"2024-09-10T18:38:37.644083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:12.713080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157509+0000","last_change":"2024-09-10T18:38:28.149166+0000","last_active":"2024-09-10T18:38:38.157509+0000","last_peered":"2024-09-10T18:38:38.157509+0000","last_clean":"2024-09-10T18:38:38.157509+0000","last_became_active":"2024-09-10T18:38:28.148903+0000","last_became_peered":"2024-09-10T18:38:28.148903+0000","last_unstale":"2024-09-10T18:38:38.157509+0000","last_undegraded":"2024-09-10T18:38:38.157509+0000","last_fullsized":"2024-09-10T18:38:38.157509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:52.697610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642434+0000","last_change":"2024-09-10T18:38:25.928407+0000","last_active":"2024-09-10T18:38:37.642434+0000","last_peered":"2024-09-10T18:38:37.642434+0000","last_clean":"2024-09-10T18:38:37.642434+0000","last_became_active":"2024-09-10T18:38:25.928188+0000","last_became_peered":"2024-09-10T18:38:25.928188+0000","last_unstale":"2024-09-10T18:38:37.642434+0000","last_undegraded":"2024-09-10T18:38:37.642434+0000","last_fullsized":"2024-09-10T18:38:37.642434+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:12.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:04.233117+0000","last_change":"2024-09-10T18:38:14.657207+0000","last_active":"2024-09-10T18:39:04.233117+0000","last_peered":"2024-09-10T18:39:04.233117+0000","last_clean":"2024-09-10T18:39:04.233117+0000","last_became_active":"2024-09-10T18:38:14.651890+0000","last_became_peered":"2024-09-10T18:38:14.651890+0000","last_unstale":"2024-09-10T18:39:04.233117+0000","last_undegraded":"2024-09-10T18:39:04.233117+0000","last_fullsized":"2024-09-10T18:39:04.233117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:36:17.003750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:36:17.003750+0000","last_clean_scrub_stamp":"2024-09-10T18:36:17.003750+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.473298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702198+0000","last_change":"2024-09-10T18:38:29.937884+0000","last_active":"2024-09-10T18:38:37.702198+0000","last_peered":"2024-09-10T18:38:37.702198+0000","last_clean":"2024-09-10T18:38:37.702198+0000","last_became_active":"2024-09-10T18:38:29.937610+0000","last_became_peered":"2024-09-10T18:38:29.937610+0000","last_unstale":"2024-09-10T18:38:37.702198+0000","last_undegraded":"2024-09-10T18:38:37.702198+0000","last_fullsized":"2024-09-10T18:38:37.702198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:19.014360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158575+0000","last_change":"2024-09-10T18:38:31.951564+0000","last_active":"2024-09-10T18:38:38.158575+0000","last_peered":"2024-09-10T18:38:38.158575+0000","last_clean":"2024-09-10T18:38:38.158575+0000","last_became_active":"2024-09-10T18:38:31.951312+0000","last_became_peered":"2024-09-10T18:38:31.951312+0000","last_unstale":"2024-09-10T18:38:38.158575+0000","last_undegraded":"2024-09-10T18:38:38.158575+0000","last_fullsized":"2024-09-10T18:38:38.158575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:48.310808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644194+0000","last_change":"2024-09-10T18:38:27.932549+0000","last_active":"2024-09-10T18:38:37.644194+0000","last_peered":"2024-09-10T18:38:37.644194+0000","last_clean":"2024-09-10T18:38:37.644194+0000","last_became_active":"2024-09-10T18:38:27.932182+0000","last_became_peered":"2024-09-10T18:38:27.932182+0000","last_unstale":"2024-09-10T18:38:37.644194+0000","last_undegraded":"2024-09-10T18:38:37.644194+0000","last_fullsized":"2024-09-10T18:38:37.644194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:14.966759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643111+0000","last_change":"2024-09-10T18:38:25.927365+0000","last_active":"2024-09-10T18:38:37.643111+0000","last_peered":"2024-09-10T18:38:37.643111+0000","last_clean":"2024-09-10T18:38:37.643111+0000","last_became_active":"2024-09-10T18:38:25.927121+0000","last_became_peered":"2024-09-10T18:38:25.927121+0000","last_unstale":"2024-09-10T18:38:37.643111+0000","last_undegraded":"2024-09-10T18:38:37.643111+0000","last_fullsized":"2024-09-10T18:38:37.643111+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:44.340897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158413+0000","last_change":"2024-09-10T18:38:29.943436+0000","last_active":"2024-09-10T18:38:38.158413+0000","last_peered":"2024-09-10T18:38:38.158413+0000","last_clean":"2024-09-10T18:38:38.158413+0000","last_became_active":"2024-09-10T18:38:29.943124+0000","last_became_peered":"2024-09-10T18:38:29.943124+0000","last_unstale":"2024-09-10T18:38:38.158413+0000","last_undegraded":"2024-09-10T18:38:38.158413+0000","last_fullsized":"2024-09-10T18:38:38.158413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:37.450134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644093+0000","last_change":"2024-09-10T18:38:32.250012+0000","last_active":"2024-09-10T18:38:37.644093+0000","last_peered":"2024-09-10T18:38:37.644093+0000","last_clean":"2024-09-10T18:38:37.644093+0000","last_became_active":"2024-09-10T18:38:32.249615+0000","last_became_peered":"2024-09-10T18:38:32.249615+0000","last_unstale":"2024-09-10T18:38:37.644093+0000","last_undegraded":"2024-09-10T18:38:37.644093+0000","last_fullsized":"2024-09-10T18:38:37.644093+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:51.594685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643557+0000","last_change":"2024-09-10T18:38:27.932612+0000","last_active":"2024-09-10T18:38:37.643557+0000","last_peered":"2024-09-10T18:38:37.643557+0000","last_clean":"2024-09-10T18:38:37.643557+0000","last_became_active":"2024-09-10T18:38:27.932350+0000","last_became_peered":"2024-09-10T18:38:27.932350+0000","last_unstale":"2024-09-10T18:38:37.643557+0000","last_undegraded":"2024-09-10T18:38:37.643557+0000","last_fullsized":"2024-09-10T18:38:37.643557+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:31.487587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157319+0000","last_change":"2024-09-10T18:38:25.928808+0000","last_active":"2024-09-10T18:38:38.157319+0000","last_peered":"2024-09-10T18:38:38.157319+0000","last_clean":"2024-09-10T18:38:38.157319+0000","last_became_active":"2024-09-10T18:38:25.928524+0000","last_became_peered":"2024-09-10T18:38:25.928524+0000","last_unstale":"2024-09-10T18:38:38.157319+0000","last_undegraded":"2024-09-10T18:38:38.157319+0000","last_fullsized":"2024-09-10T18:38:38.157319+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:38.252658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644160+0000","last_change":"2024-09-10T18:38:29.935939+0000","last_active":"2024-09-10T18:38:37.644160+0000","last_peered":"2024-09-10T18:38:37.644160+0000","last_clean":"2024-09-10T18:38:37.644160+0000","last_became_active":"2024-09-10T18:38:29.935694+0000","last_became_peered":"2024-09-10T18:38:29.935694+0000","last_unstale":"2024-09-10T18:38:37.644160+0000","last_undegraded":"2024-09-10T18:38:37.644160+0000","last_fullsized":"2024-09-10T18:38:37.644160+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:35.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157073+0000","last_change":"2024-09-10T18:38:31.950799+0000","last_active":"2024-09-10T18:38:38.157073+0000","last_peered":"2024-09-10T18:38:38.157073+0000","last_clean":"2024-09-10T18:38:38.157073+0000","last_became_active":"2024-09-10T18:38:31.950498+0000","last_became_peered":"2024-09-10T18:38:31.950498+0000","last_unstale":"2024-09-10T18:38:38.157073+0000","last_undegraded":"2024-09-10T18:38:38.157073+0000","last_fullsized":"2024-09-10T18:38:38.157073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:51.293619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643334+0000","last_change":"2024-09-10T18:38:27.928093+0000","last_active":"2024-09-10T18:38:37.643334+0000","last_peered":"2024-09-10T18:38:37.643334+0000","last_clean":"2024-09-10T18:38:37.643334+0000","last_became_active":"2024-09-10T18:38:27.927815+0000","last_became_peered":"2024-09-10T18:38:27.927815+0000","last_unstale":"2024-09-10T18:38:37.643334+0000","last_undegraded":"2024-09-10T18:38:37.643334+0000","last_fullsized":"2024-09-10T18:38:37.643334+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:43.358629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644529+0000","last_change":"2024-09-10T18:38:25.926714+0000","last_active":"2024-09-10T18:38:37.644529+0000","last_peered":"2024-09-10T18:38:37.644529+0000","last_clean":"2024-09-10T18:38:37.644529+0000","last_became_active":"2024-09-10T18:38:25.926163+0000","last_became_peered":"2024-09-10T18:38:25.926163+0000","last_unstale":"2024-09-10T18:38:37.644529+0000","last_undegraded":"2024-09-10T18:38:37.644529+0000","last_fullsized":"2024-09-10T18:38:37.644529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:19.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256608+0000","last_change":"2024-09-10T18:38:29.942153+0000","last_active":"2024-09-10T18:39:39.256608+0000","last_peered":"2024-09-10T18:39:39.256608+0000","last_clean":"2024-09-10T18:39:39.256608+0000","last_became_active":"2024-09-10T18:38:29.941866+0000","last_became_peered":"2024-09-10T18:38:29.941866+0000","last_unstale":"2024-09-10T18:39:39.256608+0000","last_undegraded":"2024-09-10T18:39:39.256608+0000","last_fullsized":"2024-09-10T18:39:39.256608+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:00.601973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643939+0000","last_change":"2024-09-10T18:38:31.947312+0000","last_active":"2024-09-10T18:38:37.643939+0000","last_peered":"2024-09-10T18:38:37.643939+0000","last_clean":"2024-09-10T18:38:37.643939+0000","last_became_active":"2024-09-10T18:38:31.947047+0000","last_became_peered":"2024-09-10T18:38:31.947047+0000","last_unstale":"2024-09-10T18:38:37.643939+0000","last_undegraded":"2024-09-10T18:38:37.643939+0000","last_fullsized":"2024-09-10T18:38:37.643939+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:24.131245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704326+0000","last_change":"2024-09-10T18:38:27.929890+0000","last_active":"2024-09-10T18:38:37.704326+0000","last_peered":"2024-09-10T18:38:37.704326+0000","last_clean":"2024-09-10T18:38:37.704326+0000","last_became_active":"2024-09-10T18:38:27.929598+0000","last_became_peered":"2024-09-10T18:38:27.929598+0000","last_unstale":"2024-09-10T18:38:37.704326+0000","last_undegraded":"2024-09-10T18:38:37.704326+0000","last_fullsized":"2024-09-10T18:38:37.704326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:27.278215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158272+0000","last_change":"2024-09-10T18:38:25.923333+0000","last_active":"2024-09-10T18:38:38.158272+0000","last_peered":"2024-09-10T18:38:38.158272+0000","last_clean":"2024-09-10T18:38:38.158272+0000","last_became_active":"2024-09-10T18:38:25.923074+0000","last_became_peered":"2024-09-10T18:38:25.923074+0000","last_unstale":"2024-09-10T18:38:38.158272+0000","last_undegraded":"2024-09-10T18:38:38.158272+0000","last_fullsized":"2024-09-10T18:38:38.158272+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:59.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156177+0000","last_change":"2024-09-10T18:38:29.938634+0000","last_active":"2024-09-10T18:38:38.156177+0000","last_peered":"2024-09-10T18:38:38.156177+0000","last_clean":"2024-09-10T18:38:38.156177+0000","last_became_active":"2024-09-10T18:38:29.938347+0000","last_became_peered":"2024-09-10T18:38:29.938347+0000","last_unstale":"2024-09-10T18:38:38.156177+0000","last_undegraded":"2024-09-10T18:38:38.156177+0000","last_fullsized":"2024-09-10T18:38:38.156177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.551451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704112+0000","last_change":"2024-09-10T18:38:31.954484+0000","last_active":"2024-09-10T18:38:37.704112+0000","last_peered":"2024-09-10T18:38:37.704112+0000","last_clean":"2024-09-10T18:38:37.704112+0000","last_became_active":"2024-09-10T18:38:31.954222+0000","last_became_peered":"2024-09-10T18:38:31.954222+0000","last_unstale":"2024-09-10T18:38:37.704112+0000","last_undegraded":"2024-09-10T18:38:37.704112+0000","last_fullsized":"2024-09-10T18:38:37.704112+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:13.521093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157088+0000","last_change":"2024-09-10T18:38:28.149258+0000","last_active":"2024-09-10T18:38:38.157088+0000","last_peered":"2024-09-10T18:38:38.157088+0000","last_clean":"2024-09-10T18:38:38.157088+0000","last_became_active":"2024-09-10T18:38:28.148976+0000","last_became_peered":"2024-09-10T18:38:28.148976+0000","last_unstale":"2024-09-10T18:38:38.157088+0000","last_undegraded":"2024-09-10T18:38:38.157088+0000","last_fullsized":"2024-09-10T18:38:38.157088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:32.324241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702685+0000","last_change":"2024-09-10T18:38:25.927018+0000","last_active":"2024-09-10T18:38:37.702685+0000","last_peered":"2024-09-10T18:38:37.702685+0000","last_clean":"2024-09-10T18:38:37.702685+0000","last_became_active":"2024-09-10T18:38:25.926626+0000","last_became_peered":"2024-09-10T18:38:25.926626+0000","last_unstale":"2024-09-10T18:38:37.702685+0000","last_undegraded":"2024-09-10T18:38:37.702685+0000","last_fullsized":"2024-09-10T18:38:37.702685+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:56.716974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156177+0000","last_change":"2024-09-10T18:38:29.936554+0000","last_active":"2024-09-10T18:38:38.156177+0000","last_peered":"2024-09-10T18:38:38.156177+0000","last_clean":"2024-09-10T18:38:38.156177+0000","last_became_active":"2024-09-10T18:38:29.936293+0000","last_became_peered":"2024-09-10T18:38:29.936293+0000","last_unstale":"2024-09-10T18:38:38.156177+0000","last_undegraded":"2024-09-10T18:38:38.156177+0000","last_fullsized":"2024-09-10T18:38:38.156177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:50.963831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702096+0000","last_change":"2024-09-10T18:38:31.951356+0000","last_active":"2024-09-10T18:38:37.702096+0000","last_peered":"2024-09-10T18:38:37.702096+0000","last_clean":"2024-09-10T18:38:37.702096+0000","last_became_active":"2024-09-10T18:38:31.951068+0000","last_became_peered":"2024-09-10T18:38:31.951068+0000","last_unstale":"2024-09-10T18:38:37.702096+0000","last_undegraded":"2024-09-10T18:38:37.702096+0000","last_fullsized":"2024-09-10T18:38:37.702096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:21.496839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642948+0000","last_change":"2024-09-10T18:38:27.933341+0000","last_active":"2024-09-10T18:38:37.642948+0000","last_peered":"2024-09-10T18:38:37.642948+0000","last_clean":"2024-09-10T18:38:37.642948+0000","last_became_active":"2024-09-10T18:38:27.933064+0000","last_became_peered":"2024-09-10T18:38:27.933064+0000","last_unstale":"2024-09-10T18:38:37.642948+0000","last_undegraded":"2024-09-10T18:38:37.642948+0000","last_fullsized":"2024-09-10T18:38:37.642948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:24.221319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702791+0000","last_change":"2024-09-10T18:38:25.927335+0000","last_active":"2024-09-10T18:38:37.702791+0000","last_peered":"2024-09-10T18:38:37.702791+0000","last_clean":"2024-09-10T18:38:37.702791+0000","last_became_active":"2024-09-10T18:38:25.927063+0000","last_became_peered":"2024-09-10T18:38:25.927063+0000","last_unstale":"2024-09-10T18:38:37.702791+0000","last_undegraded":"2024-09-10T18:38:37.702791+0000","last_fullsized":"2024-09-10T18:38:37.702791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:05.955429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156180+0000","last_change":"2024-09-10T18:38:29.937324+0000","last_active":"2024-09-10T18:38:38.156180+0000","last_peered":"2024-09-10T18:38:38.156180+0000","last_clean":"2024-09-10T18:38:38.156180+0000","last_became_active":"2024-09-10T18:38:29.937065+0000","last_became_peered":"2024-09-10T18:38:29.937065+0000","last_unstale":"2024-09-10T18:38:38.156180+0000","last_undegraded":"2024-09-10T18:38:38.156180+0000","last_fullsized":"2024-09-10T18:38:38.156180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:11.590149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642341+0000","last_change":"2024-09-10T18:38:31.951247+0000","last_active":"2024-09-10T18:38:37.642341+0000","last_peered":"2024-09-10T18:38:37.642341+0000","last_clean":"2024-09-10T18:38:37.642341+0000","last_became_active":"2024-09-10T18:38:31.950846+0000","last_became_peered":"2024-09-10T18:38:31.950846+0000","last_unstale":"2024-09-10T18:38:37.642341+0000","last_undegraded":"2024-09-10T18:38:37.642341+0000","last_fullsized":"2024-09-10T18:38:37.642341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.409025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158221+0000","last_change":"2024-09-10T18:38:28.150348+0000","last_active":"2024-09-10T18:38:38.158221+0000","last_peered":"2024-09-10T18:38:38.158221+0000","last_clean":"2024-09-10T18:38:38.158221+0000","last_became_active":"2024-09-10T18:38:28.150083+0000","last_became_peered":"2024-09-10T18:38:28.150083+0000","last_unstale":"2024-09-10T18:38:38.158221+0000","last_undegraded":"2024-09-10T18:38:38.158221+0000","last_fullsized":"2024-09-10T18:38:38.158221+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:48.828846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702726+0000","last_change":"2024-09-10T18:38:25.931960+0000","last_active":"2024-09-10T18:38:37.702726+0000","last_peered":"2024-09-10T18:38:37.702726+0000","last_clean":"2024-09-10T18:38:37.702726+0000","last_became_active":"2024-09-10T18:38:25.931651+0000","last_became_peered":"2024-09-10T18:38:25.931651+0000","last_unstale":"2024-09-10T18:38:37.702726+0000","last_undegraded":"2024-09-10T18:38:37.702726+0000","last_fullsized":"2024-09-10T18:38:37.702726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:12.255164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256661+0000","last_change":"2024-09-10T18:38:29.942366+0000","last_active":"2024-09-10T18:39:39.256661+0000","last_peered":"2024-09-10T18:39:39.256661+0000","last_clean":"2024-09-10T18:39:39.256661+0000","last_became_active":"2024-09-10T18:38:29.942066+0000","last_became_peered":"2024-09-10T18:38:29.942066+0000","last_unstale":"2024-09-10T18:39:39.256661+0000","last_undegraded":"2024-09-10T18:39:39.256661+0000","last_fullsized":"2024-09-10T18:39:39.256661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:00.787089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157278+0000","last_change":"2024-09-10T18:38:32.250043+0000","last_active":"2024-09-10T18:38:38.157278+0000","last_peered":"2024-09-10T18:38:38.157278+0000","last_clean":"2024-09-10T18:38:38.157278+0000","last_became_active":"2024-09-10T18:38:32.249710+0000","last_became_peered":"2024-09-10T18:38:32.249710+0000","last_unstale":"2024-09-10T18:38:38.157278+0000","last_undegraded":"2024-09-10T18:38:38.157278+0000","last_fullsized":"2024-09-10T18:38:38.157278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:12.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156734+0000","last_change":"2024-09-10T18:38:25.923516+0000","last_active":"2024-09-10T18:38:38.156734+0000","last_peered":"2024-09-10T18:38:38.156734+0000","last_clean":"2024-09-10T18:38:38.156734+0000","last_became_active":"2024-09-10T18:38:25.922859+0000","last_became_peered":"2024-09-10T18:38:25.922859+0000","last_unstale":"2024-09-10T18:38:38.156734+0000","last_undegraded":"2024-09-10T18:38:38.156734+0000","last_fullsized":"2024-09-10T18:38:38.156734+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:35.924757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643443+0000","last_change":"2024-09-10T18:38:28.150752+0000","last_active":"2024-09-10T18:38:37.643443+0000","last_peered":"2024-09-10T18:38:37.643443+0000","last_clean":"2024-09-10T18:38:37.643443+0000","last_became_active":"2024-09-10T18:38:28.150483+0000","last_became_peered":"2024-09-10T18:38:28.150483+0000","last_unstale":"2024-09-10T18:38:37.643443+0000","last_undegraded":"2024-09-10T18:38:37.643443+0000","last_fullsized":"2024-09-10T18:38:37.643443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:16.147142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158161+0000","last_change":"2024-09-10T18:38:29.940447+0000","last_active":"2024-09-10T18:38:38.158161+0000","last_peered":"2024-09-10T18:38:38.158161+0000","last_clean":"2024-09-10T18:38:38.158161+0000","last_became_active":"2024-09-10T18:38:29.940193+0000","last_became_peered":"2024-09-10T18:38:29.940193+0000","last_unstale":"2024-09-10T18:38:38.158161+0000","last_undegraded":"2024-09-10T18:38:38.158161+0000","last_fullsized":"2024-09-10T18:38:38.158161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:27.522578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702317+0000","last_change":"2024-09-10T18:38:32.251240+0000","last_active":"2024-09-10T18:38:37.702317+0000","last_peered":"2024-09-10T18:38:37.702317+0000","last_clean":"2024-09-10T18:38:37.702317+0000","last_became_active":"2024-09-10T18:38:32.250985+0000","last_became_peered":"2024-09-10T18:38:32.250985+0000","last_unstale":"2024-09-10T18:38:37.702317+0000","last_undegraded":"2024-09-10T18:38:37.702317+0000","last_fullsized":"2024-09-10T18:38:37.702317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:44.264429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704467+0000","last_change":"2024-09-10T18:38:28.151456+0000","last_active":"2024-09-10T18:38:37.704467+0000","last_peered":"2024-09-10T18:38:37.704467+0000","last_clean":"2024-09-10T18:38:37.704467+0000","last_became_active":"2024-09-10T18:38:28.151229+0000","last_became_peered":"2024-09-10T18:38:28.151229+0000","last_unstale":"2024-09-10T18:38:37.704467+0000","last_undegraded":"2024-09-10T18:38:37.704467+0000","last_fullsized":"2024-09-10T18:38:37.704467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:49.525214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.701982+0000","last_change":"2024-09-10T18:38:25.927162+0000","last_active":"2024-09-10T18:38:37.701982+0000","last_peered":"2024-09-10T18:38:37.701982+0000","last_clean":"2024-09-10T18:38:37.701982+0000","last_became_active":"2024-09-10T18:38:25.926889+0000","last_became_peered":"2024-09-10T18:38:25.926889+0000","last_unstale":"2024-09-10T18:38:37.701982+0000","last_undegraded":"2024-09-10T18:38:37.701982+0000","last_fullsized":"2024-09-10T18:38:37.701982+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:37.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643367+0000","last_change":"2024-09-10T18:38:29.930601+0000","last_active":"2024-09-10T18:38:37.643367+0000","last_peered":"2024-09-10T18:38:37.643367+0000","last_clean":"2024-09-10T18:38:37.643367+0000","last_became_active":"2024-09-10T18:38:29.930347+0000","last_became_peered":"2024-09-10T18:38:29.930347+0000","last_unstale":"2024-09-10T18:38:37.643367+0000","last_undegraded":"2024-09-10T18:38:37.643367+0000","last_fullsized":"2024-09-10T18:38:37.643367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156722+0000","last_change":"2024-09-10T18:38:31.947131+0000","last_active":"2024-09-10T18:38:38.156722+0000","last_peered":"2024-09-10T18:38:38.156722+0000","last_clean":"2024-09-10T18:38:38.156722+0000","last_became_active":"2024-09-10T18:38:31.946811+0000","last_became_peered":"2024-09-10T18:38:31.946811+0000","last_unstale":"2024-09-10T18:38:38.156722+0000","last_undegraded":"2024-09-10T18:38:38.156722+0000","last_fullsized":"2024-09-10T18:38:38.156722+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:58.722351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158418+0000","last_change":"2024-09-10T18:38:28.150289+0000","last_active":"2024-09-10T18:38:38.158418+0000","last_peered":"2024-09-10T18:38:38.158418+0000","last_clean":"2024-09-10T18:38:38.158418+0000","last_became_active":"2024-09-10T18:38:28.149914+0000","last_became_peered":"2024-09-10T18:38:28.149914+0000","last_unstale":"2024-09-10T18:38:38.158418+0000","last_undegraded":"2024-09-10T18:38:38.158418+0000","last_fullsized":"2024-09-10T18:38:38.158418+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:41.737642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704070+0000","last_change":"2024-09-10T18:38:25.926841+0000","last_active":"2024-09-10T18:38:37.704070+0000","last_peered":"2024-09-10T18:38:37.704070+0000","last_clean":"2024-09-10T18:38:37.704070+0000","last_became_active":"2024-09-10T18:38:25.926103+0000","last_became_peered":"2024-09-10T18:38:25.926103+0000","last_unstale":"2024-09-10T18:38:37.704070+0000","last_undegraded":"2024-09-10T18:38:37.704070+0000","last_fullsized":"2024-09-10T18:38:37.704070+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:01.354756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256847+0000","last_change":"2024-09-10T18:38:29.933279+0000","last_active":"2024-09-10T18:39:39.256847+0000","last_peered":"2024-09-10T18:39:39.256847+0000","last_clean":"2024-09-10T18:39:39.256847+0000","last_became_active":"2024-09-10T18:38:29.932998+0000","last_became_peered":"2024-09-10T18:38:29.932998+0000","last_unstale":"2024-09-10T18:39:39.256847+0000","last_undegraded":"2024-09-10T18:39:39.256847+0000","last_fullsized":"2024-09-10T18:39:39.256847+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:57.323649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643599+0000","last_change":"2024-09-10T18:38:31.943984+0000","last_active":"2024-09-10T18:38:37.643599+0000","last_peered":"2024-09-10T18:38:37.643599+0000","last_clean":"2024-09-10T18:38:37.643599+0000","last_became_active":"2024-09-10T18:38:31.943696+0000","last_became_peered":"2024-09-10T18:38:31.943696+0000","last_unstale":"2024-09-10T18:38:37.643599+0000","last_undegraded":"2024-09-10T18:38:37.643599+0000","last_fullsized":"2024-09-10T18:38:37.643599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:58.529164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644339+0000","last_change":"2024-09-10T18:38:28.149294+0000","last_active":"2024-09-10T18:38:37.644339+0000","last_peered":"2024-09-10T18:38:37.644339+0000","last_clean":"2024-09-10T18:38:37.644339+0000","last_became_active":"2024-09-10T18:38:28.149013+0000","last_became_peered":"2024-09-10T18:38:28.149013+0000","last_unstale":"2024-09-10T18:38:37.644339+0000","last_undegraded":"2024-09-10T18:38:37.644339+0000","last_fullsized":"2024-09-10T18:38:37.644339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:08.721381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702614+0000","last_change":"2024-09-10T18:38:25.930316+0000","last_active":"2024-09-10T18:38:37.702614+0000","last_peered":"2024-09-10T18:38:37.702614+0000","last_clean":"2024-09-10T18:38:37.702614+0000","last_became_active":"2024-09-10T18:38:25.930062+0000","last_became_peered":"2024-09-10T18:38:25.930062+0000","last_unstale":"2024-09-10T18:38:37.702614+0000","last_undegraded":"2024-09-10T18:38:37.702614+0000","last_fullsized":"2024-09-10T18:38:37.702614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:34.888229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:39.256578+0000","last_change":"2024-09-10T18:38:29.939458+0000","last_active":"2024-09-10T18:39:39.256578+0000","last_peered":"2024-09-10T18:39:39.256578+0000","last_clean":"2024-09-10T18:39:39.256578+0000","last_became_active":"2024-09-10T18:38:29.938483+0000","last_became_peered":"2024-09-10T18:38:29.938483+0000","last_unstale":"2024-09-10T18:39:39.256578+0000","last_undegraded":"2024-09-10T18:39:39.256578+0000","last_fullsized":"2024-09-10T18:39:39.256578+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.832330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704316+0000","last_change":"2024-09-10T18:38:31.953351+0000","last_active":"2024-09-10T18:38:37.704316+0000","last_peered":"2024-09-10T18:38:37.704316+0000","last_clean":"2024-09-10T18:38:37.704316+0000","last_became_active":"2024-09-10T18:38:31.950794+0000","last_became_peered":"2024-09-10T18:38:31.950794+0000","last_unstale":"2024-09-10T18:38:37.704316+0000","last_undegraded":"2024-09-10T18:38:37.704316+0000","last_fullsized":"2024-09-10T18:38:37.704316+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:47.667461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642964+0000","last_change":"2024-09-10T18:38:27.932852+0000","last_active":"2024-09-10T18:38:37.642964+0000","last_peered":"2024-09-10T18:38:37.642964+0000","last_clean":"2024-09-10T18:38:37.642964+0000","last_became_active":"2024-09-10T18:38:27.932098+0000","last_became_peered":"2024-09-10T18:38:27.932098+0000","last_unstale":"2024-09-10T18:38:37.642964+0000","last_undegraded":"2024-09-10T18:38:37.642964+0000","last_fullsized":"2024-09-10T18:38:37.642964+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:12.484774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643458+0000","last_change":"2024-09-10T18:38:25.926498+0000","last_active":"2024-09-10T18:38:37.643458+0000","last_peered":"2024-09-10T18:38:37.643458+0000","last_clean":"2024-09-10T18:38:37.643458+0000","last_became_active":"2024-09-10T18:38:25.926236+0000","last_became_peered":"2024-09-10T18:38:25.926236+0000","last_unstale":"2024-09-10T18:38:37.643458+0000","last_undegraded":"2024-09-10T18:38:37.643458+0000","last_fullsized":"2024-09-10T18:38:37.643458+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:43.645054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642688+0000","last_change":"2024-09-10T18:38:29.935449+0000","last_active":"2024-09-10T18:38:37.642688+0000","last_peered":"2024-09-10T18:38:37.642688+0000","last_clean":"2024-09-10T18:38:37.642688+0000","last_became_active":"2024-09-10T18:38:29.935192+0000","last_became_peered":"2024-09-10T18:38:29.935192+0000","last_unstale":"2024-09-10T18:38:37.642688+0000","last_undegraded":"2024-09-10T18:38:37.642688+0000","last_fullsized":"2024-09-10T18:38:37.642688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:46.309122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.641987+0000","last_change":"2024-09-10T18:38:31.955592+0000","last_active":"2024-09-10T18:38:37.641987+0000","last_peered":"2024-09-10T18:38:37.641987+0000","last_clean":"2024-09-10T18:38:37.641987+0000","last_became_active":"2024-09-10T18:38:31.955300+0000","last_became_peered":"2024-09-10T18:38:31.955300+0000","last_unstale":"2024-09-10T18:38:37.641987+0000","last_undegraded":"2024-09-10T18:38:37.641987+0000","last_fullsized":"2024-09-10T18:38:37.641987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:54.422302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642805+0000","last_change":"2024-09-10T18:38:27.925922+0000","last_active":"2024-09-10T18:38:37.642805+0000","last_peered":"2024-09-10T18:38:37.642805+0000","last_clean":"2024-09-10T18:38:37.642805+0000","last_became_active":"2024-09-10T18:38:27.925609+0000","last_became_peered":"2024-09-10T18:38:27.925609+0000","last_unstale":"2024-09-10T18:38:37.642805+0000","last_undegraded":"2024-09-10T18:38:37.642805+0000","last_fullsized":"2024-09-10T18:38:37.642805+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:38.690994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702836+0000","last_change":"2024-09-10T18:38:25.929704+0000","last_active":"2024-09-10T18:38:37.702836+0000","last_peered":"2024-09-10T18:38:37.702836+0000","last_clean":"2024-09-10T18:38:37.702836+0000","last_became_active":"2024-09-10T18:38:25.929406+0000","last_became_peered":"2024-09-10T18:38:25.929406+0000","last_unstale":"2024-09-10T18:38:37.702836+0000","last_undegraded":"2024-09-10T18:38:37.702836+0000","last_fullsized":"2024-09-10T18:38:37.702836+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:37.138300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644424+0000","last_change":"2024-09-10T18:38:29.936152+0000","last_active":"2024-09-10T18:38:37.644424+0000","last_peered":"2024-09-10T18:38:37.644424+0000","last_clean":"2024-09-10T18:38:37.644424+0000","last_became_active":"2024-09-10T18:38:29.935855+0000","last_became_peered":"2024-09-10T18:38:29.935855+0000","last_unstale":"2024-09-10T18:38:37.644424+0000","last_undegraded":"2024-09-10T18:38:37.644424+0000","last_fullsized":"2024-09-10T18:38:37.644424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:41.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.703800+0000","last_change":"2024-09-10T18:38:31.954284+0000","last_active":"2024-09-10T18:38:37.703800+0000","last_peered":"2024-09-10T18:38:37.703800+0000","last_clean":"2024-09-10T18:38:37.703800+0000","last_became_active":"2024-09-10T18:38:31.953890+0000","last_became_peered":"2024-09-10T18:38:31.953890+0000","last_unstale":"2024-09-10T18:38:37.703800+0000","last_undegraded":"2024-09-10T18:38:37.703800+0000","last_fullsized":"2024-09-10T18:38:37.703800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:45.251878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642106+0000","last_change":"2024-09-10T18:38:28.149263+0000","last_active":"2024-09-10T18:38:37.642106+0000","last_peered":"2024-09-10T18:38:37.642106+0000","last_clean":"2024-09-10T18:38:37.642106+0000","last_became_active":"2024-09-10T18:38:28.149006+0000","last_became_peered":"2024-09-10T18:38:28.149006+0000","last_unstale":"2024-09-10T18:38:37.642106+0000","last_undegraded":"2024-09-10T18:38:37.642106+0000","last_fullsized":"2024-09-10T18:38:37.642106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:04.800274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158682+0000","last_change":"2024-09-10T18:38:25.927561+0000","last_active":"2024-09-10T18:38:38.158682+0000","last_peered":"2024-09-10T18:38:38.158682+0000","last_clean":"2024-09-10T18:38:38.158682+0000","last_became_active":"2024-09-10T18:38:25.927280+0000","last_became_peered":"2024-09-10T18:38:25.927280+0000","last_unstale":"2024-09-10T18:38:38.158682+0000","last_undegraded":"2024-09-10T18:38:38.158682+0000","last_fullsized":"2024-09-10T18:38:38.158682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:17.616362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.701705+0000","last_change":"2024-09-10T18:38:29.935678+0000","last_active":"2024-09-10T18:38:37.701705+0000","last_peered":"2024-09-10T18:38:37.701705+0000","last_clean":"2024-09-10T18:38:37.701705+0000","last_became_active":"2024-09-10T18:38:29.935373+0000","last_became_peered":"2024-09-10T18:38:29.935373+0000","last_unstale":"2024-09-10T18:38:37.701705+0000","last_undegraded":"2024-09-10T18:38:37.701705+0000","last_fullsized":"2024-09-10T18:38:37.701705+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:37.171547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644520+0000","last_change":"2024-09-10T18:38:32.252268+0000","last_active":"2024-09-10T18:38:37.644520+0000","last_peered":"2024-09-10T18:38:37.644520+0000","last_clean":"2024-09-10T18:38:37.644520+0000","last_became_active":"2024-09-10T18:38:32.251960+0000","last_became_peered":"2024-09-10T18:38:32.251960+0000","last_unstale":"2024-09-10T18:38:37.644520+0000","last_undegraded":"2024-09-10T18:38:37.644520+0000","last_fullsized":"2024-09-10T18:38:37.644520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:24.747601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.641898+0000","last_change":"2024-09-10T18:38:27.937582+0000","last_active":"2024-09-10T18:38:37.641898+0000","last_peered":"2024-09-10T18:38:37.641898+0000","last_clean":"2024-09-10T18:38:37.641898+0000","last_became_active":"2024-09-10T18:38:27.937268+0000","last_became_peered":"2024-09-10T18:38:27.937268+0000","last_unstale":"2024-09-10T18:38:37.641898+0000","last_undegraded":"2024-09-10T18:38:37.641898+0000","last_fullsized":"2024-09-10T18:38:37.641898+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:03.469222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704529+0000","last_change":"2024-09-10T18:38:25.926901+0000","last_active":"2024-09-10T18:38:37.704529+0000","last_peered":"2024-09-10T18:38:37.704529+0000","last_clean":"2024-09-10T18:38:37.704529+0000","last_became_active":"2024-09-10T18:38:25.925436+0000","last_became_peered":"2024-09-10T18:38:25.925436+0000","last_unstale":"2024-09-10T18:38:37.704529+0000","last_undegraded":"2024-09-10T18:38:37.704529+0000","last_fullsized":"2024-09-10T18:38:37.704529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:16.142397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156878+0000","last_change":"2024-09-10T18:38:29.940012+0000","last_active":"2024-09-10T18:38:38.156878+0000","last_peered":"2024-09-10T18:38:38.156878+0000","last_clean":"2024-09-10T18:38:38.156878+0000","last_became_active":"2024-09-10T18:38:29.939680+0000","last_became_peered":"2024-09-10T18:38:29.939680+0000","last_unstale":"2024-09-10T18:38:38.156878+0000","last_undegraded":"2024-09-10T18:38:38.156878+0000","last_fullsized":"2024-09-10T18:38:38.156878+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:44.113143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.701931+0000","last_change":"2024-09-10T18:38:31.951741+0000","last_active":"2024-09-10T18:38:37.701931+0000","last_peered":"2024-09-10T18:38:37.701931+0000","last_clean":"2024-09-10T18:38:37.701931+0000","last_became_active":"2024-09-10T18:38:31.950607+0000","last_became_peered":"2024-09-10T18:38:31.950607+0000","last_unstale":"2024-09-10T18:38:37.701931+0000","last_undegraded":"2024-09-10T18:38:37.701931+0000","last_fullsized":"2024-09-10T18:38:37.701931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:25.539726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643403+0000","last_change":"2024-09-10T18:38:31.944905+0000","last_active":"2024-09-10T18:38:37.643403+0000","last_peered":"2024-09-10T18:38:37.643403+0000","last_clean":"2024-09-10T18:38:37.643403+0000","last_became_active":"2024-09-10T18:38:31.944646+0000","last_became_peered":"2024-09-10T18:38:31.944646+0000","last_unstale":"2024-09-10T18:38:37.643403+0000","last_undegraded":"2024-09-10T18:38:37.643403+0000","last_fullsized":"2024-09-10T18:38:37.643403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:18.817888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642044+0000","last_change":"2024-09-10T18:38:25.927049+0000","last_active":"2024-09-10T18:38:37.642044+0000","last_peered":"2024-09-10T18:38:37.642044+0000","last_clean":"2024-09-10T18:38:37.642044+0000","last_became_active":"2024-09-10T18:38:25.926795+0000","last_became_peered":"2024-09-10T18:38:25.926795+0000","last_unstale":"2024-09-10T18:38:37.642044+0000","last_undegraded":"2024-09-10T18:38:37.642044+0000","last_fullsized":"2024-09-10T18:38:37.642044+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:14.811208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157948+0000","last_change":"2024-09-10T18:38:27.932946+0000","last_active":"2024-09-10T18:38:38.157948+0000","last_peered":"2024-09-10T18:38:38.157948+0000","last_clean":"2024-09-10T18:38:38.157948+0000","last_became_active":"2024-09-10T18:38:27.932116+0000","last_became_peered":"2024-09-10T18:38:27.932116+0000","last_unstale":"2024-09-10T18:38:38.157948+0000","last_undegraded":"2024-09-10T18:38:38.157948+0000","last_fullsized":"2024-09-10T18:38:38.157948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:53.858520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704332+0000","last_change":"2024-09-10T18:38:29.936826+0000","last_active":"2024-09-10T18:38:37.704332+0000","last_peered":"2024-09-10T18:38:37.704332+0000","last_clean":"2024-09-10T18:38:37.704332+0000","last_became_active":"2024-09-10T18:38:29.936587+0000","last_became_peered":"2024-09-10T18:38:29.936587+0000","last_unstale":"2024-09-10T18:38:37.704332+0000","last_undegraded":"2024-09-10T18:38:37.704332+0000","last_fullsized":"2024-09-10T18:38:37.704332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:23.724546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430229,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1212416,"data_stored":823032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38100000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.33600000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.504},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.45900000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.48899999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52600000000000002}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298512,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446640,"statfs":{"total":95995035648,"available":95689359360,"internally_reserved":0,"allocated":1187840,"data_stored":821666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.50900000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.434},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.54500000000000004}]},{"osd":3,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.499}]},{"osd":4,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.497,"5min":0.497,"15min":0.497},"last":0.375},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.34200000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.40100000000000002}]}]},{"osd":5,"up_from":36,"seq":154618822687,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297932,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447220,"statfs":{"total":95995035648,"available":95689953280,"internally_reserved":0,"allocated":589824,"data_stored":231295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.63500000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50900000000000001},"min":{"1min":0.43099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.61499999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.501,"15min":0.497},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.40699999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.65000000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.436,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.44900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.64400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.59899999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.35899999999999999},"min":{"1min":0.31,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.35499999999999998},"min":{"1min":0.316,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.33000000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.34100000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.35599999999999998}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":622592,"data_stored":233349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47299999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.624,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45800000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.623,"5min":0.63,"15min":0.63},"last":0.51300000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.50700000000000001,"15min":0.504},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.505,"15min":0.503},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.442},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.498}]},{"osd":5,"last update":"Tue Sep 10 18:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35999999999999999},"min":{"1min":0.28299999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.442,"5min":0.374,"15min":0.36199999999999999},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.40300000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.38800000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.41899999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182439,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":233319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:42 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.34499999999999997,"15min":0.35899999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.44800000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38300000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.38800000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:39:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38300000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.5,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.379,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47899999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:39:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.437,"15min":0.45300000000000001},"min":{"1min":0.29499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53100000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.40600000000000003,"15min":0.41899999999999998},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.31900000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68600000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.46899999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.752,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.495},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.435},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.51300000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":231727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44400000000000001,"5min":0.496,"15min":0.496},"last":0.246},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.376,"15min":0.37},"min":{"1min":0.26000000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.33400000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.42699999999999999,"15min":0.42499999999999999},"min":{"1min":0.28699999999999998,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.44400000000000001,"15min":0.442},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.311}]},{"osd":4,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76500000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.755,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54900000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:39:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76900000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.625}]},{"osd":6,"last update":"Tue Sep 10 18:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57399999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":233655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34599999999999997,"15min":0.34300000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.5,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58899999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.30399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.22,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52000000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.318},{"interface":"front","average":{"1min":0.374,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54300000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.26800000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54400000000000004,"5min":1.593,"15min":1.593},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.33500000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:38:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47399999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.497},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.442}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1212416,"data_stored":823637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.30299999999999999},"min":{"1min":0.23799999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45800000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.32200000000000001},"min":{"1min":0.251,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.59699999999999998,"5min":1.04,"15min":1.04},"last":0.35399999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.33600000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:39:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.46100000000000002,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.43099999999999999,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.373}]},{"osd":4,"last update":"Tue Sep 10 18:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70899999999999996,"5min":1.099,"15min":1.099},"last":0.40500000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.46200000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.437},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.44800000000000001}]},{"osd":7,"last update":"Tue Sep 10 18:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.496},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:39:53.142 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:39:53.403 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 podman[77178]: 2024-09-10 18:39:53.301816103 +0000 UTC m=+4.064933548 container init ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 podman[77178]: 2024-09-10 18:39:53.306748305 +0000 UTC m=+4.069865749 container start ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.310Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.310Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.311Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.311Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.311Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:39:53.404 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:39:53.405 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:39:53.406 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:39:53.407 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:39:53.408 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:39:53.408 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:39:53.408 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:39:53.408 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a[77437]: ts=2024-09-10T18:39:53.312Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:39:53.408 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 bash[77178]: ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e 2024-09-10T18:39:53.477 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:53.613 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:39:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:39:53.710 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:39:53 smithi131 systemd[1]: Started Ceph node-exporter.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:39:53.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[29369]: from='client.14727 v1:172.21.15.131:0/2375845620' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:39:53.710 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[29369]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[40885]: from='client.14727 v1:172.21.15.131:0/2375845620' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[40885]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.711 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:53 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:53 smithi175 ceph-mon[33399]: from='client.14727 v1:172.21.15.131:0/2375845620' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:53 smithi175 ceph-mon[33399]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:53 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:53 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:53.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:53 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:54.365 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:54 smithi175 systemd[1]: Starting Ceph node-exporter.b for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:39:54.615 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:54.616 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:54 smithi175 bash[67301]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:39:54.914 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:54.914 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:54 smithi131 ceph-mon[29369]: Deploying daemon node-exporter.b on smithi175 2024-09-10T18:39:54.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:39:54.914 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:54 smithi131 ceph-mon[40885]: Deploying daemon node-exporter.b on smithi175 2024-09-10T18:39:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:54 smithi175 ceph-mon[33399]: Deploying daemon node-exporter.b on smithi175 2024-09-10T18:39:54.927 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:54 smithi175 bash[67301]: Getting image source signatures 2024-09-10T18:39:54.927 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:54 smithi175 bash[67301]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:39:54.927 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:54 smithi175 bash[67301]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:39:54.927 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:54 smithi175 bash[67301]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:39:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:39:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:39:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:39:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- 172.21.15.131:0/1628682257 >> v1:172.21.15.131:6790/0 conn(0x7faf70107e00 legacy=0x7faf7010a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:55.526 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- 172.21.15.131:0/1628682257 shutdown_connections 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- 172.21.15.131:0/1628682257 >> 172.21.15.131:0/1628682257 conn(0x7faf700752c0 msgr2=0x7faf700f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- 172.21.15.131:0/1628682257 shutdown_connections 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- 172.21.15.131:0/1628682257 wait complete. 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 Processor -- start 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- start start 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf7011bf70 con 0x7faf701042a0 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf7011d130 con 0x7faf70107e00 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7776e640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf7011e330 con 0x7faf70101530 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7550a640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7faf70101530 0x7faf70065010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:43092/0 (socket says 172.21.15.131:43092) 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf7550a640 1 -- 172.21.15.131:0/568184779 learned_addr learned my addr 172.21.15.131:0/568184779 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.496+0000 7faf74d09640 1 --1- 172.21.15.131:0/568184779 >> v1:172.21.15.131:6789/0 conn(0x7faf701042a0 0x7faf70065720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45908/0 (socket says 172.21.15.131:45908) 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888983777 0 0) 0x7faf7011e330 con 0x7faf70101530 2024-09-10T18:39:55.527 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf5c003120 con 0x7faf70101530 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177206455 0 0) 0x7faf7011bf70 con 0x7faf701042a0 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf7011e330 con 0x7faf701042a0 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687156749 0 0) 0x7faf7011d130 con 0x7faf70107e00 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf7011bf70 con 0x7faf70107e00 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1816873900 0 0) 0x7faf5c003120 con 0x7faf70101530 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf7011d130 con 0x7faf70101530 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7faf6c001980 con 0x7faf70101530 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 878746156 0 0) 0x7faf7011e330 con 0x7faf701042a0 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf5c003120 con 0x7faf701042a0 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7faf60001960 con 0x7faf701042a0 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3827419516 0 0) 0x7faf7011bf70 con 0x7faf70107e00 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf7011e330 con 0x7faf70107e00 2024-09-10T18:39:55.528 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7faf68001930 con 0x7faf70107e00 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1832717144 0 0) 0x7faf7011d130 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 >> v1:172.21.15.175:6789/0 conn(0x7faf70107e00 legacy=0x7faf7011a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 >> v1:172.21.15.131:6789/0 conn(0x7faf701042a0 legacy=0x7faf70065720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7011f530 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7011c160 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7faf6c0020b0 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7faf6c005a40 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.497+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7011c5b0 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.498+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf700fcc30 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.499+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7faf6c001c60 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.499+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7faf6c089fe0 con 0x7faf70101530 2024-09-10T18:39:55.529 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.503+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7faf6c056d30 con 0x7faf70101530 2024-09-10T18:39:55.658 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.656+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 --> v1:172.21.15.131:6800/336444945 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf7010d820 con 0x7faf5c06fef0 2024-09-10T18:39:55.666 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.665+0000 7faf65ffb640 1 -- 172.21.15.131:0/568184779 <== mgr.34496 v1:172.21.15.131:6800/336444945 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384019 (unknown 2965378022 0 2988514962) 0x7faf7010d820 con 0x7faf5c06fef0 2024-09-10T18:39:55.670 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:39:55.672 INFO:teuthology.orchestra.run.smithi131.stderr:dumped all 2024-09-10T18:39:55.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.668+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 >> v1:172.21.15.131:6800/336444945 conn(0x7faf5c06fef0 legacy=0x7faf5c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:55.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.668+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 >> v1:172.21.15.131:6790/0 conn(0x7faf70101530 legacy=0x7faf70065010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:55.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.668+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 shutdown_connections 2024-09-10T18:39:55.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.668+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 >> 172.21.15.131:0/568184779 conn(0x7faf700752c0 msgr2=0x7faf7010b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:55.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.668+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 shutdown_connections 2024-09-10T18:39:55.674 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:55.668+0000 7faf7776e640 1 -- 172.21.15.131:0/568184779 wait complete. 2024-09-10T18:39:55.679 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:55 smithi131 ceph-mon[29369]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:55.679 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:55 smithi131 ceph-mon[40885]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:55 smithi175 ceph-mon[33399]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:56.395 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 bash[67301]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:39:56.395 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 bash[67301]: Writing manifest to image destination 2024-09-10T18:39:56.465 INFO:teuthology.orchestra.run.smithi131.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T18:39:54.195441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":934,"num_read_kb":791,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385392,"kb_used_data":6548,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575824,"statfs":{"total":767960285184,"available":765517643776,"internally_reserved":0,"allocated":6705152,"data_stored":3631680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002434"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704541+0000","last_change":"2024-09-10T18:38:25.926949+0000","last_active":"2024-09-10T18:38:37.704541+0000","last_peered":"2024-09-10T18:38:37.704541+0000","last_clean":"2024-09-10T18:38:37.704541+0000","last_became_active":"2024-09-10T18:38:25.925806+0000","last_became_peered":"2024-09-10T18:38:25.925806+0000","last_unstale":"2024-09-10T18:38:37.704541+0000","last_undegraded":"2024-09-10T18:38:37.704541+0000","last_fullsized":"2024-09-10T18:38:37.704541+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:52:57.574425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642685+0000","last_change":"2024-09-10T18:38:27.933810+0000","last_active":"2024-09-10T18:38:37.642685+0000","last_peered":"2024-09-10T18:38:37.642685+0000","last_clean":"2024-09-10T18:38:37.642685+0000","last_became_active":"2024-09-10T18:38:27.933367+0000","last_became_peered":"2024-09-10T18:38:27.933367+0000","last_unstale":"2024-09-10T18:38:37.642685+0000","last_undegraded":"2024-09-10T18:38:37.642685+0000","last_fullsized":"2024-09-10T18:38:37.642685+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:53:30.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643363+0000","last_change":"2024-09-10T18:38:29.935908+0000","last_active":"2024-09-10T18:38:37.643363+0000","last_peered":"2024-09-10T18:38:37.643363+0000","last_clean":"2024-09-10T18:38:37.643363+0000","last_became_active":"2024-09-10T18:38:29.935600+0000","last_became_peered":"2024-09-10T18:38:29.935600+0000","last_unstale":"2024-09-10T18:38:37.643363+0000","last_undegraded":"2024-09-10T18:38:37.643363+0000","last_fullsized":"2024-09-10T18:38:37.643363+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:46.492384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642876+0000","last_change":"2024-09-10T18:38:31.951048+0000","last_active":"2024-09-10T18:38:37.642876+0000","last_peered":"2024-09-10T18:38:37.642876+0000","last_clean":"2024-09-10T18:38:37.642876+0000","last_became_active":"2024-09-10T18:38:31.950702+0000","last_became_peered":"2024-09-10T18:38:31.950702+0000","last_unstale":"2024-09-10T18:38:37.642876+0000","last_undegraded":"2024-09-10T18:38:37.642876+0000","last_fullsized":"2024-09-10T18:38:37.642876+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:52.911650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642258+0000","last_change":"2024-09-10T18:38:32.252268+0000","last_active":"2024-09-10T18:38:37.642258+0000","last_peered":"2024-09-10T18:38:37.642258+0000","last_clean":"2024-09-10T18:38:37.642258+0000","last_became_active":"2024-09-10T18:38:32.251977+0000","last_became_peered":"2024-09-10T18:38:32.251977+0000","last_unstale":"2024-09-10T18:38:37.642258+0000","last_undegraded":"2024-09-10T18:38:37.642258+0000","last_fullsized":"2024-09-10T18:38:37.642258+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:49:35.397136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643998+0000","last_change":"2024-09-10T18:38:25.930789+0000","last_active":"2024-09-10T18:38:37.643998+0000","last_peered":"2024-09-10T18:38:37.643998+0000","last_clean":"2024-09-10T18:38:37.643998+0000","last_became_active":"2024-09-10T18:38:25.930521+0000","last_became_peered":"2024-09-10T18:38:25.930521+0000","last_unstale":"2024-09-10T18:38:37.643998+0000","last_undegraded":"2024-09-10T18:38:37.643998+0000","last_fullsized":"2024-09-10T18:38:37.643998+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:37:33.460802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642436+0000","last_change":"2024-09-10T18:38:27.931540+0000","last_active":"2024-09-10T18:38:37.642436+0000","last_peered":"2024-09-10T18:38:37.642436+0000","last_clean":"2024-09-10T18:38:37.642436+0000","last_became_active":"2024-09-10T18:38:27.930987+0000","last_became_peered":"2024-09-10T18:38:27.930987+0000","last_unstale":"2024-09-10T18:38:37.642436+0000","last_undegraded":"2024-09-10T18:38:37.642436+0000","last_fullsized":"2024-09-10T18:38:37.642436+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:29.064086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642677+0000","last_change":"2024-09-10T18:38:29.942627+0000","last_active":"2024-09-10T18:38:37.642677+0000","last_peered":"2024-09-10T18:38:37.642677+0000","last_clean":"2024-09-10T18:38:37.642677+0000","last_became_active":"2024-09-10T18:38:29.942358+0000","last_became_peered":"2024-09-10T18:38:29.942358+0000","last_unstale":"2024-09-10T18:38:37.642677+0000","last_undegraded":"2024-09-10T18:38:37.642677+0000","last_fullsized":"2024-09-10T18:38:37.642677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:39.988568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158715+0000","last_change":"2024-09-10T18:38:25.924451+0000","last_active":"2024-09-10T18:38:38.158715+0000","last_peered":"2024-09-10T18:38:38.158715+0000","last_clean":"2024-09-10T18:38:38.158715+0000","last_became_active":"2024-09-10T18:38:25.924203+0000","last_became_peered":"2024-09-10T18:38:25.924203+0000","last_unstale":"2024-09-10T18:38:38.158715+0000","last_undegraded":"2024-09-10T18:38:38.158715+0000","last_fullsized":"2024-09-10T18:38:38.158715+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:39.727352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643163+0000","last_change":"2024-09-10T18:38:27.928758+0000","last_active":"2024-09-10T18:38:37.643163+0000","last_peered":"2024-09-10T18:38:37.643163+0000","last_clean":"2024-09-10T18:38:37.643163+0000","last_became_active":"2024-09-10T18:38:27.928561+0000","last_became_peered":"2024-09-10T18:38:27.928561+0000","last_unstale":"2024-09-10T18:38:37.643163+0000","last_undegraded":"2024-09-10T18:38:37.643163+0000","last_fullsized":"2024-09-10T18:38:37.643163+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:15.983852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157733+0000","last_change":"2024-09-10T18:38:29.942125+0000","last_active":"2024-09-10T18:38:38.157733+0000","last_peered":"2024-09-10T18:38:38.157733+0000","last_clean":"2024-09-10T18:38:38.157733+0000","last_became_active":"2024-09-10T18:38:29.941829+0000","last_became_peered":"2024-09-10T18:38:29.941829+0000","last_unstale":"2024-09-10T18:38:38.157733+0000","last_undegraded":"2024-09-10T18:38:38.157733+0000","last_fullsized":"2024-09-10T18:38:38.157733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:39:39.792363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.703948+0000","last_change":"2024-09-10T18:38:31.953741+0000","last_active":"2024-09-10T18:38:37.703948+0000","last_peered":"2024-09-10T18:38:37.703948+0000","last_clean":"2024-09-10T18:38:37.703948+0000","last_became_active":"2024-09-10T18:38:31.952446+0000","last_became_peered":"2024-09-10T18:38:31.952446+0000","last_unstale":"2024-09-10T18:38:37.703948+0000","last_undegraded":"2024-09-10T18:38:37.703948+0000","last_fullsized":"2024-09-10T18:38:37.703948+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:55.147920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158156+0000","last_change":"2024-09-10T18:38:25.926950+0000","last_active":"2024-09-10T18:38:38.158156+0000","last_peered":"2024-09-10T18:38:38.158156+0000","last_clean":"2024-09-10T18:38:38.158156+0000","last_became_active":"2024-09-10T18:38:25.926653+0000","last_became_peered":"2024-09-10T18:38:25.926653+0000","last_unstale":"2024-09-10T18:38:38.158156+0000","last_undegraded":"2024-09-10T18:38:38.158156+0000","last_fullsized":"2024-09-10T18:38:38.158156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:56.063659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642876+0000","last_change":"2024-09-10T18:38:27.933795+0000","last_active":"2024-09-10T18:38:37.642876+0000","last_peered":"2024-09-10T18:38:37.642876+0000","last_clean":"2024-09-10T18:38:37.642876+0000","last_became_active":"2024-09-10T18:38:27.933358+0000","last_became_peered":"2024-09-10T18:38:27.933358+0000","last_unstale":"2024-09-10T18:38:37.642876+0000","last_undegraded":"2024-09-10T18:38:37.642876+0000","last_fullsized":"2024-09-10T18:38:37.642876+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:50.231995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702017+0000","last_change":"2024-09-10T18:38:29.938986+0000","last_active":"2024-09-10T18:38:37.702017+0000","last_peered":"2024-09-10T18:38:37.702017+0000","last_clean":"2024-09-10T18:38:37.702017+0000","last_became_active":"2024-09-10T18:38:29.938656+0000","last_became_peered":"2024-09-10T18:38:29.938656+0000","last_unstale":"2024-09-10T18:38:37.702017+0000","last_undegraded":"2024-09-10T18:38:37.702017+0000","last_fullsized":"2024-09-10T18:38:37.702017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:59.184333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156974+0000","last_change":"2024-09-10T18:38:31.950912+0000","last_active":"2024-09-10T18:38:38.156974+0000","last_peered":"2024-09-10T18:38:38.156974+0000","last_clean":"2024-09-10T18:38:38.156974+0000","last_became_active":"2024-09-10T18:38:31.950631+0000","last_became_peered":"2024-09-10T18:38:31.950631+0000","last_unstale":"2024-09-10T18:38:38.156974+0000","last_undegraded":"2024-09-10T18:38:38.156974+0000","last_fullsized":"2024-09-10T18:38:38.156974+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:21:33.589815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644018+0000","last_change":"2024-09-10T18:38:32.250423+0000","last_active":"2024-09-10T18:38:37.644018+0000","last_peered":"2024-09-10T18:38:37.644018+0000","last_clean":"2024-09-10T18:38:37.644018+0000","last_became_active":"2024-09-10T18:38:32.250138+0000","last_became_peered":"2024-09-10T18:38:32.250138+0000","last_unstale":"2024-09-10T18:38:37.644018+0000","last_undegraded":"2024-09-10T18:38:37.644018+0000","last_fullsized":"2024-09-10T18:38:37.644018+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:37:04.859620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704511+0000","last_change":"2024-09-10T18:38:25.924952+0000","last_active":"2024-09-10T18:38:37.704511+0000","last_peered":"2024-09-10T18:38:37.704511+0000","last_clean":"2024-09-10T18:38:37.704511+0000","last_became_active":"2024-09-10T18:38:25.924666+0000","last_became_peered":"2024-09-10T18:38:25.924666+0000","last_unstale":"2024-09-10T18:38:37.704511+0000","last_undegraded":"2024-09-10T18:38:37.704511+0000","last_fullsized":"2024-09-10T18:38:37.704511+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:01.282908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157054+0000","last_change":"2024-09-10T18:38:27.930615+0000","last_active":"2024-09-10T18:38:38.157054+0000","last_peered":"2024-09-10T18:38:38.157054+0000","last_clean":"2024-09-10T18:38:38.157054+0000","last_became_active":"2024-09-10T18:38:27.930326+0000","last_became_peered":"2024-09-10T18:38:27.930326+0000","last_unstale":"2024-09-10T18:38:38.157054+0000","last_undegraded":"2024-09-10T18:38:38.157054+0000","last_fullsized":"2024-09-10T18:38:38.157054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:55.729441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643648+0000","last_change":"2024-09-10T18:38:29.935741+0000","last_active":"2024-09-10T18:38:37.643648+0000","last_peered":"2024-09-10T18:38:37.643648+0000","last_clean":"2024-09-10T18:38:37.643648+0000","last_became_active":"2024-09-10T18:38:29.935034+0000","last_became_peered":"2024-09-10T18:38:29.935034+0000","last_unstale":"2024-09-10T18:38:37.643648+0000","last_undegraded":"2024-09-10T18:38:37.643648+0000","last_fullsized":"2024-09-10T18:38:37.643648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:06.651867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643300+0000","last_change":"2024-09-10T18:38:32.249832+0000","last_active":"2024-09-10T18:38:37.643300+0000","last_peered":"2024-09-10T18:38:37.643300+0000","last_clean":"2024-09-10T18:38:37.643300+0000","last_became_active":"2024-09-10T18:38:32.249492+0000","last_became_peered":"2024-09-10T18:38:32.249492+0000","last_unstale":"2024-09-10T18:38:37.643300+0000","last_undegraded":"2024-09-10T18:38:37.643300+0000","last_fullsized":"2024-09-10T18:38:37.643300+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:39.245093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643231+0000","last_change":"2024-09-10T18:38:25.931692+0000","last_active":"2024-09-10T18:38:37.643231+0000","last_peered":"2024-09-10T18:38:37.643231+0000","last_clean":"2024-09-10T18:38:37.643231+0000","last_became_active":"2024-09-10T18:38:25.930675+0000","last_became_peered":"2024-09-10T18:38:25.930675+0000","last_unstale":"2024-09-10T18:38:37.643231+0000","last_undegraded":"2024-09-10T18:38:37.643231+0000","last_fullsized":"2024-09-10T18:38:37.643231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:13.164100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643008+0000","last_change":"2024-09-10T18:38:27.937292+0000","last_active":"2024-09-10T18:38:37.643008+0000","last_peered":"2024-09-10T18:38:37.643008+0000","last_clean":"2024-09-10T18:38:37.643008+0000","last_became_active":"2024-09-10T18:38:27.936980+0000","last_became_peered":"2024-09-10T18:38:27.936980+0000","last_unstale":"2024-09-10T18:38:37.643008+0000","last_undegraded":"2024-09-10T18:38:37.643008+0000","last_fullsized":"2024-09-10T18:38:37.643008+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:59:35.116701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643328+0000","last_change":"2024-09-10T18:38:29.940856+0000","last_active":"2024-09-10T18:38:37.643328+0000","last_peered":"2024-09-10T18:38:37.643328+0000","last_clean":"2024-09-10T18:38:37.643328+0000","last_became_active":"2024-09-10T18:38:29.940581+0000","last_became_peered":"2024-09-10T18:38:29.940581+0000","last_unstale":"2024-09-10T18:38:37.643328+0000","last_undegraded":"2024-09-10T18:38:37.643328+0000","last_fullsized":"2024-09-10T18:38:37.643328+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.764961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702029+0000","last_change":"2024-09-10T18:38:31.950648+0000","last_active":"2024-09-10T18:38:37.702029+0000","last_peered":"2024-09-10T18:38:37.702029+0000","last_clean":"2024-09-10T18:38:37.702029+0000","last_became_active":"2024-09-10T18:38:31.950370+0000","last_became_peered":"2024-09-10T18:38:31.950370+0000","last_unstale":"2024-09-10T18:38:37.702029+0000","last_undegraded":"2024-09-10T18:38:37.702029+0000","last_fullsized":"2024-09-10T18:38:37.702029+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:07.922850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642995+0000","last_change":"2024-09-10T18:38:25.924183+0000","last_active":"2024-09-10T18:38:37.642995+0000","last_peered":"2024-09-10T18:38:37.642995+0000","last_clean":"2024-09-10T18:38:37.642995+0000","last_became_active":"2024-09-10T18:38:25.923881+0000","last_became_peered":"2024-09-10T18:38:25.923881+0000","last_unstale":"2024-09-10T18:38:37.642995+0000","last_undegraded":"2024-09-10T18:38:37.642995+0000","last_fullsized":"2024-09-10T18:38:37.642995+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:19:17.583082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704407+0000","last_change":"2024-09-10T18:38:28.150726+0000","last_active":"2024-09-10T18:38:37.704407+0000","last_peered":"2024-09-10T18:38:37.704407+0000","last_clean":"2024-09-10T18:38:37.704407+0000","last_became_active":"2024-09-10T18:38:28.150471+0000","last_became_peered":"2024-09-10T18:38:28.150471+0000","last_unstale":"2024-09-10T18:38:37.704407+0000","last_undegraded":"2024-09-10T18:38:37.704407+0000","last_fullsized":"2024-09-10T18:38:37.704407+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:44.263683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257529+0000","last_change":"2024-09-10T18:38:29.940452+0000","last_active":"2024-09-10T18:39:44.257529+0000","last_peered":"2024-09-10T18:39:44.257529+0000","last_clean":"2024-09-10T18:39:44.257529+0000","last_became_active":"2024-09-10T18:38:29.939059+0000","last_became_peered":"2024-09-10T18:38:29.939059+0000","last_unstale":"2024-09-10T18:39:44.257529+0000","last_undegraded":"2024-09-10T18:39:44.257529+0000","last_fullsized":"2024-09-10T18:39:44.257529+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:40:39.971086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644138+0000","last_change":"2024-09-10T18:38:27.935949+0000","last_active":"2024-09-10T18:38:37.644138+0000","last_peered":"2024-09-10T18:38:37.644138+0000","last_clean":"2024-09-10T18:38:37.644138+0000","last_became_active":"2024-09-10T18:38:27.935674+0000","last_became_peered":"2024-09-10T18:38:27.935674+0000","last_unstale":"2024-09-10T18:38:37.644138+0000","last_undegraded":"2024-09-10T18:38:37.644138+0000","last_fullsized":"2024-09-10T18:38:37.644138+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:17.374429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642044+0000","last_change":"2024-09-10T18:38:25.929533+0000","last_active":"2024-09-10T18:38:37.642044+0000","last_peered":"2024-09-10T18:38:37.642044+0000","last_clean":"2024-09-10T18:38:37.642044+0000","last_became_active":"2024-09-10T18:38:25.929214+0000","last_became_peered":"2024-09-10T18:38:25.929214+0000","last_unstale":"2024-09-10T18:38:37.642044+0000","last_undegraded":"2024-09-10T18:38:37.642044+0000","last_fullsized":"2024-09-10T18:38:37.642044+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:43.870184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257556+0000","last_change":"2024-09-10T18:38:29.943705+0000","last_active":"2024-09-10T18:39:44.257556+0000","last_peered":"2024-09-10T18:39:44.257556+0000","last_clean":"2024-09-10T18:39:44.257556+0000","last_became_active":"2024-09-10T18:38:29.943424+0000","last_became_peered":"2024-09-10T18:38:29.943424+0000","last_unstale":"2024-09-10T18:39:44.257556+0000","last_undegraded":"2024-09-10T18:39:44.257556+0000","last_fullsized":"2024-09-10T18:39:44.257556+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:12:36.965628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158152+0000","last_change":"2024-09-10T18:38:31.950934+0000","last_active":"2024-09-10T18:38:38.158152+0000","last_peered":"2024-09-10T18:38:38.158152+0000","last_clean":"2024-09-10T18:38:38.158152+0000","last_became_active":"2024-09-10T18:38:31.950664+0000","last_became_peered":"2024-09-10T18:38:31.950664+0000","last_unstale":"2024-09-10T18:38:38.158152+0000","last_undegraded":"2024-09-10T18:38:38.158152+0000","last_fullsized":"2024-09-10T18:38:38.158152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:02.893403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.641936+0000","last_change":"2024-09-10T18:38:28.150754+0000","last_active":"2024-09-10T18:38:37.641936+0000","last_peered":"2024-09-10T18:38:37.641936+0000","last_clean":"2024-09-10T18:38:37.641936+0000","last_became_active":"2024-09-10T18:38:28.150484+0000","last_became_peered":"2024-09-10T18:38:28.150484+0000","last_unstale":"2024-09-10T18:38:37.641936+0000","last_undegraded":"2024-09-10T18:38:37.641936+0000","last_fullsized":"2024-09-10T18:38:37.641936+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:46.312788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643614+0000","last_change":"2024-09-10T18:38:25.926310+0000","last_active":"2024-09-10T18:38:37.643614+0000","last_peered":"2024-09-10T18:38:37.643614+0000","last_clean":"2024-09-10T18:38:37.643614+0000","last_became_active":"2024-09-10T18:38:25.925756+0000","last_became_peered":"2024-09-10T18:38:25.925756+0000","last_unstale":"2024-09-10T18:38:37.643614+0000","last_undegraded":"2024-09-10T18:38:37.643614+0000","last_fullsized":"2024-09-10T18:38:37.643614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:35.132143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642707+0000","last_change":"2024-09-10T18:38:29.942601+0000","last_active":"2024-09-10T18:38:37.642707+0000","last_peered":"2024-09-10T18:38:37.642707+0000","last_clean":"2024-09-10T18:38:37.642707+0000","last_became_active":"2024-09-10T18:38:29.942125+0000","last_became_peered":"2024-09-10T18:38:29.942125+0000","last_unstale":"2024-09-10T18:38:37.642707+0000","last_undegraded":"2024-09-10T18:38:37.642707+0000","last_fullsized":"2024-09-10T18:38:37.642707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:37.094077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643091+0000","last_change":"2024-09-10T18:38:31.951970+0000","last_active":"2024-09-10T18:38:37.643091+0000","last_peered":"2024-09-10T18:38:37.643091+0000","last_clean":"2024-09-10T18:38:37.643091+0000","last_became_active":"2024-09-10T18:38:31.951643+0000","last_became_peered":"2024-09-10T18:38:31.951643+0000","last_unstale":"2024-09-10T18:38:37.643091+0000","last_undegraded":"2024-09-10T18:38:37.643091+0000","last_fullsized":"2024-09-10T18:38:37.643091+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:35.211919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643246+0000","last_change":"2024-09-10T18:38:27.925972+0000","last_active":"2024-09-10T18:38:37.643246+0000","last_peered":"2024-09-10T18:38:37.643246+0000","last_clean":"2024-09-10T18:38:37.643246+0000","last_became_active":"2024-09-10T18:38:27.925684+0000","last_became_peered":"2024-09-10T18:38:27.925684+0000","last_unstale":"2024-09-10T18:38:37.643246+0000","last_undegraded":"2024-09-10T18:38:37.643246+0000","last_fullsized":"2024-09-10T18:38:37.643246+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:34:41.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704523+0000","last_change":"2024-09-10T18:38:25.927668+0000","last_active":"2024-09-10T18:38:37.704523+0000","last_peered":"2024-09-10T18:38:37.704523+0000","last_clean":"2024-09-10T18:38:37.704523+0000","last_became_active":"2024-09-10T18:38:25.927255+0000","last_became_peered":"2024-09-10T18:38:25.927255+0000","last_unstale":"2024-09-10T18:38:37.704523+0000","last_undegraded":"2024-09-10T18:38:37.704523+0000","last_fullsized":"2024-09-10T18:38:37.704523+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:57:59.560489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257518+0000","last_change":"2024-09-10T18:38:29.938812+0000","last_active":"2024-09-10T18:39:44.257518+0000","last_peered":"2024-09-10T18:39:44.257518+0000","last_clean":"2024-09-10T18:39:44.257518+0000","last_became_active":"2024-09-10T18:38:29.938529+0000","last_became_peered":"2024-09-10T18:38:29.938529+0000","last_unstale":"2024-09-10T18:39:44.257518+0000","last_undegraded":"2024-09-10T18:39:44.257518+0000","last_fullsized":"2024-09-10T18:39:44.257518+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:22.174174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702102+0000","last_change":"2024-09-10T18:38:32.250387+0000","last_active":"2024-09-10T18:38:37.702102+0000","last_peered":"2024-09-10T18:38:37.702102+0000","last_clean":"2024-09-10T18:38:37.702102+0000","last_became_active":"2024-09-10T18:38:32.250109+0000","last_became_peered":"2024-09-10T18:38:32.250109+0000","last_unstale":"2024-09-10T18:38:37.702102+0000","last_undegraded":"2024-09-10T18:38:37.702102+0000","last_fullsized":"2024-09-10T18:38:37.702102+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:28:42.853793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642782+0000","last_change":"2024-09-10T18:38:27.932642+0000","last_active":"2024-09-10T18:38:37.642782+0000","last_peered":"2024-09-10T18:38:37.642782+0000","last_clean":"2024-09-10T18:38:37.642782+0000","last_became_active":"2024-09-10T18:38:27.932162+0000","last_became_peered":"2024-09-10T18:38:27.932162+0000","last_unstale":"2024-09-10T18:38:37.642782+0000","last_undegraded":"2024-09-10T18:38:37.642782+0000","last_fullsized":"2024-09-10T18:38:37.642782+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:49:59.099416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158724+0000","last_change":"2024-09-10T18:38:25.929666+0000","last_active":"2024-09-10T18:38:38.158724+0000","last_peered":"2024-09-10T18:38:38.158724+0000","last_clean":"2024-09-10T18:38:38.158724+0000","last_became_active":"2024-09-10T18:38:25.929395+0000","last_became_peered":"2024-09-10T18:38:25.929395+0000","last_unstale":"2024-09-10T18:38:38.158724+0000","last_undegraded":"2024-09-10T18:38:38.158724+0000","last_fullsized":"2024-09-10T18:38:38.158724+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:46:46.085963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":42,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257378+0000","last_change":"2024-09-10T18:38:29.939121+0000","last_active":"2024-09-10T18:39:44.257378+0000","last_peered":"2024-09-10T18:39:44.257378+0000","last_clean":"2024-09-10T18:39:44.257378+0000","last_became_active":"2024-09-10T18:38:29.938836+0000","last_became_peered":"2024-09-10T18:38:29.938836+0000","last_unstale":"2024-09-10T18:39:44.257378+0000","last_undegraded":"2024-09-10T18:39:44.257378+0000","last_fullsized":"2024-09-10T18:39:44.257378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:03.303855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704074+0000","last_change":"2024-09-10T18:38:31.953554+0000","last_active":"2024-09-10T18:38:37.704074+0000","last_peered":"2024-09-10T18:38:37.704074+0000","last_clean":"2024-09-10T18:38:37.704074+0000","last_became_active":"2024-09-10T18:38:31.951868+0000","last_became_peered":"2024-09-10T18:38:31.951868+0000","last_unstale":"2024-09-10T18:38:37.704074+0000","last_undegraded":"2024-09-10T18:38:37.704074+0000","last_fullsized":"2024-09-10T18:38:37.704074+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:02:02.862015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704339+0000","last_change":"2024-09-10T18:38:28.149464+0000","last_active":"2024-09-10T18:38:37.704339+0000","last_peered":"2024-09-10T18:38:37.704339+0000","last_clean":"2024-09-10T18:38:37.704339+0000","last_became_active":"2024-09-10T18:38:28.149177+0000","last_became_peered":"2024-09-10T18:38:28.149177+0000","last_unstale":"2024-09-10T18:38:37.704339+0000","last_undegraded":"2024-09-10T18:38:37.704339+0000","last_fullsized":"2024-09-10T18:38:37.704339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:28.427327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644484+0000","last_change":"2024-09-10T18:38:25.927418+0000","last_active":"2024-09-10T18:38:37.644484+0000","last_peered":"2024-09-10T18:38:37.644484+0000","last_clean":"2024-09-10T18:38:37.644484+0000","last_became_active":"2024-09-10T18:38:25.927115+0000","last_became_peered":"2024-09-10T18:38:25.927115+0000","last_unstale":"2024-09-10T18:38:37.644484+0000","last_undegraded":"2024-09-10T18:38:37.644484+0000","last_fullsized":"2024-09-10T18:38:37.644484+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:29.778298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156929+0000","last_change":"2024-09-10T18:38:29.937895+0000","last_active":"2024-09-10T18:38:38.156929+0000","last_peered":"2024-09-10T18:38:38.156929+0000","last_clean":"2024-09-10T18:38:38.156929+0000","last_became_active":"2024-09-10T18:38:29.937631+0000","last_became_peered":"2024-09-10T18:38:29.937631+0000","last_unstale":"2024-09-10T18:38:38.156929+0000","last_undegraded":"2024-09-10T18:38:38.156929+0000","last_fullsized":"2024-09-10T18:38:38.156929+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:23.858281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642633+0000","last_change":"2024-09-10T18:38:32.250040+0000","last_active":"2024-09-10T18:38:37.642633+0000","last_peered":"2024-09-10T18:38:37.642633+0000","last_clean":"2024-09-10T18:38:37.642633+0000","last_became_active":"2024-09-10T18:38:32.249721+0000","last_became_peered":"2024-09-10T18:38:32.249721+0000","last_unstale":"2024-09-10T18:38:37.642633+0000","last_undegraded":"2024-09-10T18:38:37.642633+0000","last_fullsized":"2024-09-10T18:38:37.642633+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:28.491717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":81,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257378+0000","last_change":"2024-09-10T18:38:27.933681+0000","last_active":"2024-09-10T18:39:44.257378+0000","last_peered":"2024-09-10T18:39:44.257378+0000","last_clean":"2024-09-10T18:39:44.257378+0000","last_became_active":"2024-09-10T18:38:27.933424+0000","last_became_peered":"2024-09-10T18:38:27.933424+0000","last_unstale":"2024-09-10T18:39:44.257378+0000","last_undegraded":"2024-09-10T18:39:44.257378+0000","last_fullsized":"2024-09-10T18:39:44.257378+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:13.041194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643205+0000","last_change":"2024-09-10T18:38:25.922397+0000","last_active":"2024-09-10T18:38:37.643205+0000","last_peered":"2024-09-10T18:38:37.643205+0000","last_clean":"2024-09-10T18:38:37.643205+0000","last_became_active":"2024-09-10T18:38:25.922119+0000","last_became_peered":"2024-09-10T18:38:25.922119+0000","last_unstale":"2024-09-10T18:38:37.643205+0000","last_undegraded":"2024-09-10T18:38:37.643205+0000","last_fullsized":"2024-09-10T18:38:37.643205+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:32:09.092791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157586+0000","last_change":"2024-09-10T18:38:29.937535+0000","last_active":"2024-09-10T18:38:38.157586+0000","last_peered":"2024-09-10T18:38:38.157586+0000","last_clean":"2024-09-10T18:38:38.157586+0000","last_became_active":"2024-09-10T18:38:25.926644+0000","last_became_peered":"2024-09-10T18:38:25.926644+0000","last_unstale":"2024-09-10T18:38:38.157586+0000","last_undegraded":"2024-09-10T18:38:38.157586+0000","last_fullsized":"2024-09-10T18:38:38.157586+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:45:55.764411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0027062990000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704332+0000","last_change":"2024-09-10T18:38:29.934156+0000","last_active":"2024-09-10T18:38:37.704332+0000","last_peered":"2024-09-10T18:38:37.704332+0000","last_clean":"2024-09-10T18:38:37.704332+0000","last_became_active":"2024-09-10T18:38:29.933852+0000","last_became_peered":"2024-09-10T18:38:29.933852+0000","last_unstale":"2024-09-10T18:38:37.704332+0000","last_undegraded":"2024-09-10T18:38:37.704332+0000","last_fullsized":"2024-09-10T18:38:37.704332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:26.921552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643807+0000","last_change":"2024-09-10T18:38:31.951149+0000","last_active":"2024-09-10T18:38:37.643807+0000","last_peered":"2024-09-10T18:38:37.643807+0000","last_clean":"2024-09-10T18:38:37.643807+0000","last_became_active":"2024-09-10T18:38:31.950873+0000","last_became_peered":"2024-09-10T18:38:31.950873+0000","last_unstale":"2024-09-10T18:38:37.643807+0000","last_undegraded":"2024-09-10T18:38:37.643807+0000","last_fullsized":"2024-09-10T18:38:37.643807+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:54.867493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644202+0000","last_change":"2024-09-10T18:38:27.929798+0000","last_active":"2024-09-10T18:38:37.644202+0000","last_peered":"2024-09-10T18:38:37.644202+0000","last_clean":"2024-09-10T18:38:37.644202+0000","last_became_active":"2024-09-10T18:38:27.929406+0000","last_became_peered":"2024-09-10T18:38:27.929406+0000","last_unstale":"2024-09-10T18:38:37.644202+0000","last_undegraded":"2024-09-10T18:38:37.644202+0000","last_fullsized":"2024-09-10T18:38:37.644202+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:33.690844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643043+0000","last_change":"2024-09-10T18:38:25.927708+0000","last_active":"2024-09-10T18:38:37.643043+0000","last_peered":"2024-09-10T18:38:37.643043+0000","last_clean":"2024-09-10T18:38:37.643043+0000","last_became_active":"2024-09-10T18:38:25.927326+0000","last_became_peered":"2024-09-10T18:38:25.927326+0000","last_unstale":"2024-09-10T18:38:37.643043+0000","last_undegraded":"2024-09-10T18:38:37.643043+0000","last_fullsized":"2024-09-10T18:38:37.643043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:32:43.064328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156603+0000","last_change":"2024-09-10T18:38:29.929189+0000","last_active":"2024-09-10T18:38:38.156603+0000","last_peered":"2024-09-10T18:38:38.156603+0000","last_clean":"2024-09-10T18:38:38.156603+0000","last_became_active":"2024-09-10T18:38:25.930264+0000","last_became_peered":"2024-09-10T18:38:25.930264+0000","last_unstale":"2024-09-10T18:38:38.156603+0000","last_undegraded":"2024-09-10T18:38:38.156603+0000","last_fullsized":"2024-09-10T18:38:38.156603+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:54.242883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048772099999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156241+0000","last_change":"2024-09-10T18:38:29.942328+0000","last_active":"2024-09-10T18:38:38.156241+0000","last_peered":"2024-09-10T18:38:38.156241+0000","last_clean":"2024-09-10T18:38:38.156241+0000","last_became_active":"2024-09-10T18:38:29.942081+0000","last_became_peered":"2024-09-10T18:38:29.942081+0000","last_unstale":"2024-09-10T18:38:38.156241+0000","last_undegraded":"2024-09-10T18:38:38.156241+0000","last_fullsized":"2024-09-10T18:38:38.156241+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:22.535057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702344+0000","last_change":"2024-09-10T18:38:32.250727+0000","last_active":"2024-09-10T18:38:37.702344+0000","last_peered":"2024-09-10T18:38:37.702344+0000","last_clean":"2024-09-10T18:38:37.702344+0000","last_became_active":"2024-09-10T18:38:32.250462+0000","last_became_peered":"2024-09-10T18:38:32.250462+0000","last_unstale":"2024-09-10T18:38:37.702344+0000","last_undegraded":"2024-09-10T18:38:37.702344+0000","last_fullsized":"2024-09-10T18:38:37.702344+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:51:51.779636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704427+0000","last_change":"2024-09-10T18:38:27.931445+0000","last_active":"2024-09-10T18:38:37.704427+0000","last_peered":"2024-09-10T18:38:37.704427+0000","last_clean":"2024-09-10T18:38:37.704427+0000","last_became_active":"2024-09-10T18:38:27.931185+0000","last_became_peered":"2024-09-10T18:38:27.931185+0000","last_unstale":"2024-09-10T18:38:37.704427+0000","last_undegraded":"2024-09-10T18:38:37.704427+0000","last_fullsized":"2024-09-10T18:38:37.704427+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:08.329500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"52'1","reported_seq":27,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704175+0000","last_change":"2024-09-10T18:38:25.924336+0000","last_active":"2024-09-10T18:38:37.704175+0000","last_peered":"2024-09-10T18:38:37.704175+0000","last_clean":"2024-09-10T18:38:37.704175+0000","last_became_active":"2024-09-10T18:38:25.924073+0000","last_became_peered":"2024-09-10T18:38:25.924073+0000","last_unstale":"2024-09-10T18:38:37.704175+0000","last_undegraded":"2024-09-10T18:38:37.704175+0000","last_fullsized":"2024-09-10T18:38:37.704175+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:24.459110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"60'5","reported_seq":107,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:51.831062+0000","last_change":"2024-09-10T18:38:29.927353+0000","last_active":"2024-09-10T18:39:51.831062+0000","last_peered":"2024-09-10T18:39:51.831062+0000","last_clean":"2024-09-10T18:39:51.831062+0000","last_became_active":"2024-09-10T18:38:25.928223+0000","last_became_peered":"2024-09-10T18:38:25.928223+0000","last_unstale":"2024-09-10T18:39:51.831062+0000","last_undegraded":"2024-09-10T18:39:51.831062+0000","last_fullsized":"2024-09-10T18:39:51.831062+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:51.337256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039067000000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":81,"num_read_kb":76,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158322+0000","last_change":"2024-09-10T18:38:29.941619+0000","last_active":"2024-09-10T18:38:38.158322+0000","last_peered":"2024-09-10T18:38:38.158322+0000","last_clean":"2024-09-10T18:38:38.158322+0000","last_became_active":"2024-09-10T18:38:29.941242+0000","last_became_peered":"2024-09-10T18:38:29.941242+0000","last_unstale":"2024-09-10T18:38:38.158322+0000","last_undegraded":"2024-09-10T18:38:38.158322+0000","last_fullsized":"2024-09-10T18:38:38.158322+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:09.101368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644083+0000","last_change":"2024-09-10T18:38:31.951193+0000","last_active":"2024-09-10T18:38:37.644083+0000","last_peered":"2024-09-10T18:38:37.644083+0000","last_clean":"2024-09-10T18:38:37.644083+0000","last_became_active":"2024-09-10T18:38:31.950872+0000","last_became_peered":"2024-09-10T18:38:31.950872+0000","last_unstale":"2024-09-10T18:38:37.644083+0000","last_undegraded":"2024-09-10T18:38:37.644083+0000","last_fullsized":"2024-09-10T18:38:37.644083+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:12.713080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157509+0000","last_change":"2024-09-10T18:38:28.149166+0000","last_active":"2024-09-10T18:38:38.157509+0000","last_peered":"2024-09-10T18:38:38.157509+0000","last_clean":"2024-09-10T18:38:38.157509+0000","last_became_active":"2024-09-10T18:38:28.148903+0000","last_became_peered":"2024-09-10T18:38:28.148903+0000","last_unstale":"2024-09-10T18:38:38.157509+0000","last_undegraded":"2024-09-10T18:38:38.157509+0000","last_fullsized":"2024-09-10T18:38:38.157509+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:52.697610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642434+0000","last_change":"2024-09-10T18:38:25.928407+0000","last_active":"2024-09-10T18:38:37.642434+0000","last_peered":"2024-09-10T18:38:37.642434+0000","last_clean":"2024-09-10T18:38:37.642434+0000","last_became_active":"2024-09-10T18:38:25.928188+0000","last_became_peered":"2024-09-10T18:38:25.928188+0000","last_unstale":"2024-09-10T18:38:37.642434+0000","last_undegraded":"2024-09-10T18:38:37.642434+0000","last_fullsized":"2024-09-10T18:38:37.642434+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:12.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:04.233117+0000","last_change":"2024-09-10T18:38:14.657207+0000","last_active":"2024-09-10T18:39:04.233117+0000","last_peered":"2024-09-10T18:39:04.233117+0000","last_clean":"2024-09-10T18:39:04.233117+0000","last_became_active":"2024-09-10T18:38:14.651890+0000","last_became_peered":"2024-09-10T18:38:14.651890+0000","last_unstale":"2024-09-10T18:39:04.233117+0000","last_undegraded":"2024-09-10T18:39:04.233117+0000","last_fullsized":"2024-09-10T18:39:04.233117+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:36:17.003750+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:36:17.003750+0000","last_clean_scrub_stamp":"2024-09-10T18:36:17.003750+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:42.473298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702198+0000","last_change":"2024-09-10T18:38:29.937884+0000","last_active":"2024-09-10T18:38:37.702198+0000","last_peered":"2024-09-10T18:38:37.702198+0000","last_clean":"2024-09-10T18:38:37.702198+0000","last_became_active":"2024-09-10T18:38:29.937610+0000","last_became_peered":"2024-09-10T18:38:29.937610+0000","last_unstale":"2024-09-10T18:38:37.702198+0000","last_undegraded":"2024-09-10T18:38:37.702198+0000","last_fullsized":"2024-09-10T18:38:37.702198+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:19.014360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158575+0000","last_change":"2024-09-10T18:38:31.951564+0000","last_active":"2024-09-10T18:38:38.158575+0000","last_peered":"2024-09-10T18:38:38.158575+0000","last_clean":"2024-09-10T18:38:38.158575+0000","last_became_active":"2024-09-10T18:38:31.951312+0000","last_became_peered":"2024-09-10T18:38:31.951312+0000","last_unstale":"2024-09-10T18:38:38.158575+0000","last_undegraded":"2024-09-10T18:38:38.158575+0000","last_fullsized":"2024-09-10T18:38:38.158575+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:48.310808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644194+0000","last_change":"2024-09-10T18:38:27.932549+0000","last_active":"2024-09-10T18:38:37.644194+0000","last_peered":"2024-09-10T18:38:37.644194+0000","last_clean":"2024-09-10T18:38:37.644194+0000","last_became_active":"2024-09-10T18:38:27.932182+0000","last_became_peered":"2024-09-10T18:38:27.932182+0000","last_unstale":"2024-09-10T18:38:37.644194+0000","last_undegraded":"2024-09-10T18:38:37.644194+0000","last_fullsized":"2024-09-10T18:38:37.644194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:14.966759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643111+0000","last_change":"2024-09-10T18:38:25.927365+0000","last_active":"2024-09-10T18:38:37.643111+0000","last_peered":"2024-09-10T18:38:37.643111+0000","last_clean":"2024-09-10T18:38:37.643111+0000","last_became_active":"2024-09-10T18:38:25.927121+0000","last_became_peered":"2024-09-10T18:38:25.927121+0000","last_unstale":"2024-09-10T18:38:37.643111+0000","last_undegraded":"2024-09-10T18:38:37.643111+0000","last_fullsized":"2024-09-10T18:38:37.643111+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:44.340897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158413+0000","last_change":"2024-09-10T18:38:29.943436+0000","last_active":"2024-09-10T18:38:38.158413+0000","last_peered":"2024-09-10T18:38:38.158413+0000","last_clean":"2024-09-10T18:38:38.158413+0000","last_became_active":"2024-09-10T18:38:29.943124+0000","last_became_peered":"2024-09-10T18:38:29.943124+0000","last_unstale":"2024-09-10T18:38:38.158413+0000","last_undegraded":"2024-09-10T18:38:38.158413+0000","last_fullsized":"2024-09-10T18:38:38.158413+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:39:37.450134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644093+0000","last_change":"2024-09-10T18:38:32.250012+0000","last_active":"2024-09-10T18:38:37.644093+0000","last_peered":"2024-09-10T18:38:37.644093+0000","last_clean":"2024-09-10T18:38:37.644093+0000","last_became_active":"2024-09-10T18:38:32.249615+0000","last_became_peered":"2024-09-10T18:38:32.249615+0000","last_unstale":"2024-09-10T18:38:37.644093+0000","last_undegraded":"2024-09-10T18:38:37.644093+0000","last_fullsized":"2024-09-10T18:38:37.644093+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:51.594685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643557+0000","last_change":"2024-09-10T18:38:27.932612+0000","last_active":"2024-09-10T18:38:37.643557+0000","last_peered":"2024-09-10T18:38:37.643557+0000","last_clean":"2024-09-10T18:38:37.643557+0000","last_became_active":"2024-09-10T18:38:27.932350+0000","last_became_peered":"2024-09-10T18:38:27.932350+0000","last_unstale":"2024-09-10T18:38:37.643557+0000","last_undegraded":"2024-09-10T18:38:37.643557+0000","last_fullsized":"2024-09-10T18:38:37.643557+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:31.487587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157319+0000","last_change":"2024-09-10T18:38:25.928808+0000","last_active":"2024-09-10T18:38:38.157319+0000","last_peered":"2024-09-10T18:38:38.157319+0000","last_clean":"2024-09-10T18:38:38.157319+0000","last_became_active":"2024-09-10T18:38:25.928524+0000","last_became_peered":"2024-09-10T18:38:25.928524+0000","last_unstale":"2024-09-10T18:38:38.157319+0000","last_undegraded":"2024-09-10T18:38:38.157319+0000","last_fullsized":"2024-09-10T18:38:38.157319+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:16:38.252658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644160+0000","last_change":"2024-09-10T18:38:29.935939+0000","last_active":"2024-09-10T18:38:37.644160+0000","last_peered":"2024-09-10T18:38:37.644160+0000","last_clean":"2024-09-10T18:38:37.644160+0000","last_became_active":"2024-09-10T18:38:29.935694+0000","last_became_peered":"2024-09-10T18:38:29.935694+0000","last_unstale":"2024-09-10T18:38:37.644160+0000","last_undegraded":"2024-09-10T18:38:37.644160+0000","last_fullsized":"2024-09-10T18:38:37.644160+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:59:35.755337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157073+0000","last_change":"2024-09-10T18:38:31.950799+0000","last_active":"2024-09-10T18:38:38.157073+0000","last_peered":"2024-09-10T18:38:38.157073+0000","last_clean":"2024-09-10T18:38:38.157073+0000","last_became_active":"2024-09-10T18:38:31.950498+0000","last_became_peered":"2024-09-10T18:38:31.950498+0000","last_unstale":"2024-09-10T18:38:38.157073+0000","last_undegraded":"2024-09-10T18:38:38.157073+0000","last_fullsized":"2024-09-10T18:38:38.157073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:51.293619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643334+0000","last_change":"2024-09-10T18:38:27.928093+0000","last_active":"2024-09-10T18:38:37.643334+0000","last_peered":"2024-09-10T18:38:37.643334+0000","last_clean":"2024-09-10T18:38:37.643334+0000","last_became_active":"2024-09-10T18:38:27.927815+0000","last_became_peered":"2024-09-10T18:38:27.927815+0000","last_unstale":"2024-09-10T18:38:37.643334+0000","last_undegraded":"2024-09-10T18:38:37.643334+0000","last_fullsized":"2024-09-10T18:38:37.643334+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:43.358629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644529+0000","last_change":"2024-09-10T18:38:25.926714+0000","last_active":"2024-09-10T18:38:37.644529+0000","last_peered":"2024-09-10T18:38:37.644529+0000","last_clean":"2024-09-10T18:38:37.644529+0000","last_became_active":"2024-09-10T18:38:25.926163+0000","last_became_peered":"2024-09-10T18:38:25.926163+0000","last_unstale":"2024-09-10T18:38:37.644529+0000","last_undegraded":"2024-09-10T18:38:37.644529+0000","last_fullsized":"2024-09-10T18:38:37.644529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:01:19.867421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257378+0000","last_change":"2024-09-10T18:38:29.942153+0000","last_active":"2024-09-10T18:39:44.257378+0000","last_peered":"2024-09-10T18:39:44.257378+0000","last_clean":"2024-09-10T18:39:44.257378+0000","last_became_active":"2024-09-10T18:38:29.941866+0000","last_became_peered":"2024-09-10T18:38:29.941866+0000","last_unstale":"2024-09-10T18:39:44.257378+0000","last_undegraded":"2024-09-10T18:39:44.257378+0000","last_fullsized":"2024-09-10T18:39:44.257378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:00.601973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643939+0000","last_change":"2024-09-10T18:38:31.947312+0000","last_active":"2024-09-10T18:38:37.643939+0000","last_peered":"2024-09-10T18:38:37.643939+0000","last_clean":"2024-09-10T18:38:37.643939+0000","last_became_active":"2024-09-10T18:38:31.947047+0000","last_became_peered":"2024-09-10T18:38:31.947047+0000","last_unstale":"2024-09-10T18:38:37.643939+0000","last_undegraded":"2024-09-10T18:38:37.643939+0000","last_fullsized":"2024-09-10T18:38:37.643939+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:24.131245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704326+0000","last_change":"2024-09-10T18:38:27.929890+0000","last_active":"2024-09-10T18:38:37.704326+0000","last_peered":"2024-09-10T18:38:37.704326+0000","last_clean":"2024-09-10T18:38:37.704326+0000","last_became_active":"2024-09-10T18:38:27.929598+0000","last_became_peered":"2024-09-10T18:38:27.929598+0000","last_unstale":"2024-09-10T18:38:37.704326+0000","last_undegraded":"2024-09-10T18:38:37.704326+0000","last_fullsized":"2024-09-10T18:38:37.704326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:27.278215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158272+0000","last_change":"2024-09-10T18:38:25.923333+0000","last_active":"2024-09-10T18:38:38.158272+0000","last_peered":"2024-09-10T18:38:38.158272+0000","last_clean":"2024-09-10T18:38:38.158272+0000","last_became_active":"2024-09-10T18:38:25.923074+0000","last_became_peered":"2024-09-10T18:38:25.923074+0000","last_unstale":"2024-09-10T18:38:38.158272+0000","last_undegraded":"2024-09-10T18:38:38.158272+0000","last_fullsized":"2024-09-10T18:38:38.158272+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:02:59.922170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156177+0000","last_change":"2024-09-10T18:38:29.938634+0000","last_active":"2024-09-10T18:38:38.156177+0000","last_peered":"2024-09-10T18:38:38.156177+0000","last_clean":"2024-09-10T18:38:38.156177+0000","last_became_active":"2024-09-10T18:38:29.938347+0000","last_became_peered":"2024-09-10T18:38:29.938347+0000","last_unstale":"2024-09-10T18:38:38.156177+0000","last_undegraded":"2024-09-10T18:38:38.156177+0000","last_fullsized":"2024-09-10T18:38:38.156177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:48.551451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704112+0000","last_change":"2024-09-10T18:38:31.954484+0000","last_active":"2024-09-10T18:38:37.704112+0000","last_peered":"2024-09-10T18:38:37.704112+0000","last_clean":"2024-09-10T18:38:37.704112+0000","last_became_active":"2024-09-10T18:38:31.954222+0000","last_became_peered":"2024-09-10T18:38:31.954222+0000","last_unstale":"2024-09-10T18:38:37.704112+0000","last_undegraded":"2024-09-10T18:38:37.704112+0000","last_fullsized":"2024-09-10T18:38:37.704112+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:13.521093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157088+0000","last_change":"2024-09-10T18:38:28.149258+0000","last_active":"2024-09-10T18:38:38.157088+0000","last_peered":"2024-09-10T18:38:38.157088+0000","last_clean":"2024-09-10T18:38:38.157088+0000","last_became_active":"2024-09-10T18:38:28.148976+0000","last_became_peered":"2024-09-10T18:38:28.148976+0000","last_unstale":"2024-09-10T18:38:38.157088+0000","last_undegraded":"2024-09-10T18:38:38.157088+0000","last_fullsized":"2024-09-10T18:38:38.157088+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:32.324241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702685+0000","last_change":"2024-09-10T18:38:25.927018+0000","last_active":"2024-09-10T18:38:37.702685+0000","last_peered":"2024-09-10T18:38:37.702685+0000","last_clean":"2024-09-10T18:38:37.702685+0000","last_became_active":"2024-09-10T18:38:25.926626+0000","last_became_peered":"2024-09-10T18:38:25.926626+0000","last_unstale":"2024-09-10T18:38:37.702685+0000","last_undegraded":"2024-09-10T18:38:37.702685+0000","last_fullsized":"2024-09-10T18:38:37.702685+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:13:56.716974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156177+0000","last_change":"2024-09-10T18:38:29.936554+0000","last_active":"2024-09-10T18:38:38.156177+0000","last_peered":"2024-09-10T18:38:38.156177+0000","last_clean":"2024-09-10T18:38:38.156177+0000","last_became_active":"2024-09-10T18:38:29.936293+0000","last_became_peered":"2024-09-10T18:38:29.936293+0000","last_unstale":"2024-09-10T18:38:38.156177+0000","last_undegraded":"2024-09-10T18:38:38.156177+0000","last_fullsized":"2024-09-10T18:38:38.156177+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:50.963831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702096+0000","last_change":"2024-09-10T18:38:31.951356+0000","last_active":"2024-09-10T18:38:37.702096+0000","last_peered":"2024-09-10T18:38:37.702096+0000","last_clean":"2024-09-10T18:38:37.702096+0000","last_became_active":"2024-09-10T18:38:31.951068+0000","last_became_peered":"2024-09-10T18:38:31.951068+0000","last_unstale":"2024-09-10T18:38:37.702096+0000","last_undegraded":"2024-09-10T18:38:37.702096+0000","last_fullsized":"2024-09-10T18:38:37.702096+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:26:21.496839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642948+0000","last_change":"2024-09-10T18:38:27.933341+0000","last_active":"2024-09-10T18:38:37.642948+0000","last_peered":"2024-09-10T18:38:37.642948+0000","last_clean":"2024-09-10T18:38:37.642948+0000","last_became_active":"2024-09-10T18:38:27.933064+0000","last_became_peered":"2024-09-10T18:38:27.933064+0000","last_unstale":"2024-09-10T18:38:37.642948+0000","last_undegraded":"2024-09-10T18:38:37.642948+0000","last_fullsized":"2024-09-10T18:38:37.642948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:25:24.221319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702791+0000","last_change":"2024-09-10T18:38:25.927335+0000","last_active":"2024-09-10T18:38:37.702791+0000","last_peered":"2024-09-10T18:38:37.702791+0000","last_clean":"2024-09-10T18:38:37.702791+0000","last_became_active":"2024-09-10T18:38:25.927063+0000","last_became_peered":"2024-09-10T18:38:25.927063+0000","last_unstale":"2024-09-10T18:38:37.702791+0000","last_undegraded":"2024-09-10T18:38:37.702791+0000","last_fullsized":"2024-09-10T18:38:37.702791+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:05.955429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156180+0000","last_change":"2024-09-10T18:38:29.937324+0000","last_active":"2024-09-10T18:38:38.156180+0000","last_peered":"2024-09-10T18:38:38.156180+0000","last_clean":"2024-09-10T18:38:38.156180+0000","last_became_active":"2024-09-10T18:38:29.937065+0000","last_became_peered":"2024-09-10T18:38:29.937065+0000","last_unstale":"2024-09-10T18:38:38.156180+0000","last_undegraded":"2024-09-10T18:38:38.156180+0000","last_fullsized":"2024-09-10T18:38:38.156180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:54:11.590149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642341+0000","last_change":"2024-09-10T18:38:31.951247+0000","last_active":"2024-09-10T18:38:37.642341+0000","last_peered":"2024-09-10T18:38:37.642341+0000","last_clean":"2024-09-10T18:38:37.642341+0000","last_became_active":"2024-09-10T18:38:31.950846+0000","last_became_peered":"2024-09-10T18:38:31.950846+0000","last_unstale":"2024-09-10T18:38:37.642341+0000","last_undegraded":"2024-09-10T18:38:37.642341+0000","last_fullsized":"2024-09-10T18:38:37.642341+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:41.409025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158221+0000","last_change":"2024-09-10T18:38:28.150348+0000","last_active":"2024-09-10T18:38:38.158221+0000","last_peered":"2024-09-10T18:38:38.158221+0000","last_clean":"2024-09-10T18:38:38.158221+0000","last_became_active":"2024-09-10T18:38:28.150083+0000","last_became_peered":"2024-09-10T18:38:28.150083+0000","last_unstale":"2024-09-10T18:38:38.158221+0000","last_undegraded":"2024-09-10T18:38:38.158221+0000","last_fullsized":"2024-09-10T18:38:38.158221+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:48.828846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"52'2","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702726+0000","last_change":"2024-09-10T18:38:25.931960+0000","last_active":"2024-09-10T18:38:37.702726+0000","last_peered":"2024-09-10T18:38:37.702726+0000","last_clean":"2024-09-10T18:38:37.702726+0000","last_became_active":"2024-09-10T18:38:25.931651+0000","last_became_peered":"2024-09-10T18:38:25.931651+0000","last_unstale":"2024-09-10T18:38:37.702726+0000","last_undegraded":"2024-09-10T18:38:37.702726+0000","last_fullsized":"2024-09-10T18:38:37.702726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:08:12.255164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257552+0000","last_change":"2024-09-10T18:38:29.942366+0000","last_active":"2024-09-10T18:39:44.257552+0000","last_peered":"2024-09-10T18:39:44.257552+0000","last_clean":"2024-09-10T18:39:44.257552+0000","last_became_active":"2024-09-10T18:38:29.942066+0000","last_became_peered":"2024-09-10T18:38:29.942066+0000","last_unstale":"2024-09-10T18:39:44.257552+0000","last_undegraded":"2024-09-10T18:39:44.257552+0000","last_fullsized":"2024-09-10T18:39:44.257552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:00.787089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157278+0000","last_change":"2024-09-10T18:38:32.250043+0000","last_active":"2024-09-10T18:38:38.157278+0000","last_peered":"2024-09-10T18:38:38.157278+0000","last_clean":"2024-09-10T18:38:38.157278+0000","last_became_active":"2024-09-10T18:38:32.249710+0000","last_became_peered":"2024-09-10T18:38:32.249710+0000","last_unstale":"2024-09-10T18:38:38.157278+0000","last_undegraded":"2024-09-10T18:38:38.157278+0000","last_fullsized":"2024-09-10T18:38:38.157278+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:12.421726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156734+0000","last_change":"2024-09-10T18:38:25.923516+0000","last_active":"2024-09-10T18:38:38.156734+0000","last_peered":"2024-09-10T18:38:38.156734+0000","last_clean":"2024-09-10T18:38:38.156734+0000","last_became_active":"2024-09-10T18:38:25.922859+0000","last_became_peered":"2024-09-10T18:38:25.922859+0000","last_unstale":"2024-09-10T18:38:38.156734+0000","last_undegraded":"2024-09-10T18:38:38.156734+0000","last_fullsized":"2024-09-10T18:38:38.156734+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:35.924757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643443+0000","last_change":"2024-09-10T18:38:28.150752+0000","last_active":"2024-09-10T18:38:37.643443+0000","last_peered":"2024-09-10T18:38:37.643443+0000","last_clean":"2024-09-10T18:38:37.643443+0000","last_became_active":"2024-09-10T18:38:28.150483+0000","last_became_peered":"2024-09-10T18:38:28.150483+0000","last_unstale":"2024-09-10T18:38:37.643443+0000","last_undegraded":"2024-09-10T18:38:37.643443+0000","last_fullsized":"2024-09-10T18:38:37.643443+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:16.147142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158161+0000","last_change":"2024-09-10T18:38:29.940447+0000","last_active":"2024-09-10T18:38:38.158161+0000","last_peered":"2024-09-10T18:38:38.158161+0000","last_clean":"2024-09-10T18:38:38.158161+0000","last_became_active":"2024-09-10T18:38:29.940193+0000","last_became_peered":"2024-09-10T18:38:29.940193+0000","last_unstale":"2024-09-10T18:38:38.158161+0000","last_undegraded":"2024-09-10T18:38:38.158161+0000","last_fullsized":"2024-09-10T18:38:38.158161+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:27.522578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702317+0000","last_change":"2024-09-10T18:38:32.251240+0000","last_active":"2024-09-10T18:38:37.702317+0000","last_peered":"2024-09-10T18:38:37.702317+0000","last_clean":"2024-09-10T18:38:37.702317+0000","last_became_active":"2024-09-10T18:38:32.250985+0000","last_became_peered":"2024-09-10T18:38:32.250985+0000","last_unstale":"2024-09-10T18:38:37.702317+0000","last_undegraded":"2024-09-10T18:38:37.702317+0000","last_fullsized":"2024-09-10T18:38:37.702317+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:20:44.264429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704467+0000","last_change":"2024-09-10T18:38:28.151456+0000","last_active":"2024-09-10T18:38:37.704467+0000","last_peered":"2024-09-10T18:38:37.704467+0000","last_clean":"2024-09-10T18:38:37.704467+0000","last_became_active":"2024-09-10T18:38:28.151229+0000","last_became_peered":"2024-09-10T18:38:28.151229+0000","last_unstale":"2024-09-10T18:38:37.704467+0000","last_undegraded":"2024-09-10T18:38:37.704467+0000","last_fullsized":"2024-09-10T18:38:37.704467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:49.525214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.701982+0000","last_change":"2024-09-10T18:38:25.927162+0000","last_active":"2024-09-10T18:38:37.701982+0000","last_peered":"2024-09-10T18:38:37.701982+0000","last_clean":"2024-09-10T18:38:37.701982+0000","last_became_active":"2024-09-10T18:38:25.926889+0000","last_became_peered":"2024-09-10T18:38:25.926889+0000","last_unstale":"2024-09-10T18:38:37.701982+0000","last_undegraded":"2024-09-10T18:38:37.701982+0000","last_fullsized":"2024-09-10T18:38:37.701982+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:34:37.729325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643367+0000","last_change":"2024-09-10T18:38:29.930601+0000","last_active":"2024-09-10T18:38:37.643367+0000","last_peered":"2024-09-10T18:38:37.643367+0000","last_clean":"2024-09-10T18:38:37.643367+0000","last_became_active":"2024-09-10T18:38:29.930347+0000","last_became_peered":"2024-09-10T18:38:29.930347+0000","last_unstale":"2024-09-10T18:38:37.643367+0000","last_undegraded":"2024-09-10T18:38:37.643367+0000","last_fullsized":"2024-09-10T18:38:37.643367+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:25.545621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156722+0000","last_change":"2024-09-10T18:38:31.947131+0000","last_active":"2024-09-10T18:38:38.156722+0000","last_peered":"2024-09-10T18:38:38.156722+0000","last_clean":"2024-09-10T18:38:38.156722+0000","last_became_active":"2024-09-10T18:38:31.946811+0000","last_became_peered":"2024-09-10T18:38:31.946811+0000","last_unstale":"2024-09-10T18:38:38.156722+0000","last_undegraded":"2024-09-10T18:38:38.156722+0000","last_fullsized":"2024-09-10T18:38:38.156722+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:58.722351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158418+0000","last_change":"2024-09-10T18:38:28.150289+0000","last_active":"2024-09-10T18:38:38.158418+0000","last_peered":"2024-09-10T18:38:38.158418+0000","last_clean":"2024-09-10T18:38:38.158418+0000","last_became_active":"2024-09-10T18:38:28.149914+0000","last_became_peered":"2024-09-10T18:38:28.149914+0000","last_unstale":"2024-09-10T18:38:38.158418+0000","last_undegraded":"2024-09-10T18:38:38.158418+0000","last_fullsized":"2024-09-10T18:38:38.158418+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:41.737642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704070+0000","last_change":"2024-09-10T18:38:25.926841+0000","last_active":"2024-09-10T18:38:37.704070+0000","last_peered":"2024-09-10T18:38:37.704070+0000","last_clean":"2024-09-10T18:38:37.704070+0000","last_became_active":"2024-09-10T18:38:25.926103+0000","last_became_peered":"2024-09-10T18:38:25.926103+0000","last_unstale":"2024-09-10T18:38:37.704070+0000","last_undegraded":"2024-09-10T18:38:37.704070+0000","last_fullsized":"2024-09-10T18:38:37.704070+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:01.354756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257481+0000","last_change":"2024-09-10T18:38:29.933279+0000","last_active":"2024-09-10T18:39:44.257481+0000","last_peered":"2024-09-10T18:39:44.257481+0000","last_clean":"2024-09-10T18:39:44.257481+0000","last_became_active":"2024-09-10T18:38:29.932998+0000","last_became_peered":"2024-09-10T18:38:29.932998+0000","last_unstale":"2024-09-10T18:39:44.257481+0000","last_undegraded":"2024-09-10T18:39:44.257481+0000","last_fullsized":"2024-09-10T18:39:44.257481+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:57.323649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643599+0000","last_change":"2024-09-10T18:38:31.943984+0000","last_active":"2024-09-10T18:38:37.643599+0000","last_peered":"2024-09-10T18:38:37.643599+0000","last_clean":"2024-09-10T18:38:37.643599+0000","last_became_active":"2024-09-10T18:38:31.943696+0000","last_became_peered":"2024-09-10T18:38:31.943696+0000","last_unstale":"2024-09-10T18:38:37.643599+0000","last_undegraded":"2024-09-10T18:38:37.643599+0000","last_fullsized":"2024-09-10T18:38:37.643599+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:28:58.529164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644339+0000","last_change":"2024-09-10T18:38:28.149294+0000","last_active":"2024-09-10T18:38:37.644339+0000","last_peered":"2024-09-10T18:38:37.644339+0000","last_clean":"2024-09-10T18:38:37.644339+0000","last_became_active":"2024-09-10T18:38:28.149013+0000","last_became_peered":"2024-09-10T18:38:28.149013+0000","last_unstale":"2024-09-10T18:38:37.644339+0000","last_undegraded":"2024-09-10T18:38:37.644339+0000","last_fullsized":"2024-09-10T18:38:37.644339+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:02:08.721381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702614+0000","last_change":"2024-09-10T18:38:25.930316+0000","last_active":"2024-09-10T18:38:37.702614+0000","last_peered":"2024-09-10T18:38:37.702614+0000","last_clean":"2024-09-10T18:38:37.702614+0000","last_became_active":"2024-09-10T18:38:25.930062+0000","last_became_peered":"2024-09-10T18:38:25.930062+0000","last_unstale":"2024-09-10T18:38:37.702614+0000","last_undegraded":"2024-09-10T18:38:37.702614+0000","last_fullsized":"2024-09-10T18:38:37.702614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:34.888229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":39,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:39:44.257541+0000","last_change":"2024-09-10T18:38:29.939458+0000","last_active":"2024-09-10T18:39:44.257541+0000","last_peered":"2024-09-10T18:39:44.257541+0000","last_clean":"2024-09-10T18:39:44.257541+0000","last_became_active":"2024-09-10T18:38:29.938483+0000","last_became_peered":"2024-09-10T18:38:29.938483+0000","last_unstale":"2024-09-10T18:39:44.257541+0000","last_undegraded":"2024-09-10T18:39:44.257541+0000","last_fullsized":"2024-09-10T18:39:44.257541+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:49:33.832330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704316+0000","last_change":"2024-09-10T18:38:31.953351+0000","last_active":"2024-09-10T18:38:37.704316+0000","last_peered":"2024-09-10T18:38:37.704316+0000","last_clean":"2024-09-10T18:38:37.704316+0000","last_became_active":"2024-09-10T18:38:31.950794+0000","last_became_peered":"2024-09-10T18:38:31.950794+0000","last_unstale":"2024-09-10T18:38:37.704316+0000","last_undegraded":"2024-09-10T18:38:37.704316+0000","last_fullsized":"2024-09-10T18:38:37.704316+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:47.667461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642964+0000","last_change":"2024-09-10T18:38:27.932852+0000","last_active":"2024-09-10T18:38:37.642964+0000","last_peered":"2024-09-10T18:38:37.642964+0000","last_clean":"2024-09-10T18:38:37.642964+0000","last_became_active":"2024-09-10T18:38:27.932098+0000","last_became_peered":"2024-09-10T18:38:27.932098+0000","last_unstale":"2024-09-10T18:38:37.642964+0000","last_undegraded":"2024-09-10T18:38:37.642964+0000","last_fullsized":"2024-09-10T18:38:37.642964+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:12.484774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643458+0000","last_change":"2024-09-10T18:38:25.926498+0000","last_active":"2024-09-10T18:38:37.643458+0000","last_peered":"2024-09-10T18:38:37.643458+0000","last_clean":"2024-09-10T18:38:37.643458+0000","last_became_active":"2024-09-10T18:38:25.926236+0000","last_became_peered":"2024-09-10T18:38:25.926236+0000","last_unstale":"2024-09-10T18:38:37.643458+0000","last_undegraded":"2024-09-10T18:38:37.643458+0000","last_fullsized":"2024-09-10T18:38:37.643458+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:43.645054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642688+0000","last_change":"2024-09-10T18:38:29.935449+0000","last_active":"2024-09-10T18:38:37.642688+0000","last_peered":"2024-09-10T18:38:37.642688+0000","last_clean":"2024-09-10T18:38:37.642688+0000","last_became_active":"2024-09-10T18:38:29.935192+0000","last_became_peered":"2024-09-10T18:38:29.935192+0000","last_unstale":"2024-09-10T18:38:37.642688+0000","last_undegraded":"2024-09-10T18:38:37.642688+0000","last_fullsized":"2024-09-10T18:38:37.642688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:52:46.309122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.641987+0000","last_change":"2024-09-10T18:38:31.955592+0000","last_active":"2024-09-10T18:38:37.641987+0000","last_peered":"2024-09-10T18:38:37.641987+0000","last_clean":"2024-09-10T18:38:37.641987+0000","last_became_active":"2024-09-10T18:38:31.955300+0000","last_became_peered":"2024-09-10T18:38:31.955300+0000","last_unstale":"2024-09-10T18:38:37.641987+0000","last_undegraded":"2024-09-10T18:38:37.641987+0000","last_fullsized":"2024-09-10T18:38:37.641987+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:54.422302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642805+0000","last_change":"2024-09-10T18:38:27.925922+0000","last_active":"2024-09-10T18:38:37.642805+0000","last_peered":"2024-09-10T18:38:37.642805+0000","last_clean":"2024-09-10T18:38:37.642805+0000","last_became_active":"2024-09-10T18:38:27.925609+0000","last_became_peered":"2024-09-10T18:38:27.925609+0000","last_unstale":"2024-09-10T18:38:37.642805+0000","last_undegraded":"2024-09-10T18:38:37.642805+0000","last_fullsized":"2024-09-10T18:38:37.642805+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:38.690994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.702836+0000","last_change":"2024-09-10T18:38:25.929704+0000","last_active":"2024-09-10T18:38:37.702836+0000","last_peered":"2024-09-10T18:38:37.702836+0000","last_clean":"2024-09-10T18:38:37.702836+0000","last_became_active":"2024-09-10T18:38:25.929406+0000","last_became_peered":"2024-09-10T18:38:25.929406+0000","last_unstale":"2024-09-10T18:38:37.702836+0000","last_undegraded":"2024-09-10T18:38:37.702836+0000","last_fullsized":"2024-09-10T18:38:37.702836+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:37.138300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644424+0000","last_change":"2024-09-10T18:38:29.936152+0000","last_active":"2024-09-10T18:38:37.644424+0000","last_peered":"2024-09-10T18:38:37.644424+0000","last_clean":"2024-09-10T18:38:37.644424+0000","last_became_active":"2024-09-10T18:38:29.935855+0000","last_became_peered":"2024-09-10T18:38:29.935855+0000","last_unstale":"2024-09-10T18:38:37.644424+0000","last_undegraded":"2024-09-10T18:38:37.644424+0000","last_fullsized":"2024-09-10T18:38:37.644424+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:20:41.703858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.703800+0000","last_change":"2024-09-10T18:38:31.954284+0000","last_active":"2024-09-10T18:38:37.703800+0000","last_peered":"2024-09-10T18:38:37.703800+0000","last_clean":"2024-09-10T18:38:37.703800+0000","last_became_active":"2024-09-10T18:38:31.953890+0000","last_became_peered":"2024-09-10T18:38:31.953890+0000","last_unstale":"2024-09-10T18:38:37.703800+0000","last_undegraded":"2024-09-10T18:38:37.703800+0000","last_fullsized":"2024-09-10T18:38:37.703800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:42:45.251878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642106+0000","last_change":"2024-09-10T18:38:28.149263+0000","last_active":"2024-09-10T18:38:37.642106+0000","last_peered":"2024-09-10T18:38:37.642106+0000","last_clean":"2024-09-10T18:38:37.642106+0000","last_became_active":"2024-09-10T18:38:28.149006+0000","last_became_peered":"2024-09-10T18:38:28.149006+0000","last_unstale":"2024-09-10T18:38:37.642106+0000","last_undegraded":"2024-09-10T18:38:37.642106+0000","last_fullsized":"2024-09-10T18:38:37.642106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:04.800274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.158682+0000","last_change":"2024-09-10T18:38:25.927561+0000","last_active":"2024-09-10T18:38:38.158682+0000","last_peered":"2024-09-10T18:38:38.158682+0000","last_clean":"2024-09-10T18:38:38.158682+0000","last_became_active":"2024-09-10T18:38:25.927280+0000","last_became_peered":"2024-09-10T18:38:25.927280+0000","last_unstale":"2024-09-10T18:38:38.158682+0000","last_undegraded":"2024-09-10T18:38:38.158682+0000","last_fullsized":"2024-09-10T18:38:38.158682+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:17.616362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.701705+0000","last_change":"2024-09-10T18:38:29.935678+0000","last_active":"2024-09-10T18:38:37.701705+0000","last_peered":"2024-09-10T18:38:37.701705+0000","last_clean":"2024-09-10T18:38:37.701705+0000","last_became_active":"2024-09-10T18:38:29.935373+0000","last_became_peered":"2024-09-10T18:38:29.935373+0000","last_unstale":"2024-09-10T18:38:37.701705+0000","last_undegraded":"2024-09-10T18:38:37.701705+0000","last_fullsized":"2024-09-10T18:38:37.701705+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:37.171547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.644520+0000","last_change":"2024-09-10T18:38:32.252268+0000","last_active":"2024-09-10T18:38:37.644520+0000","last_peered":"2024-09-10T18:38:37.644520+0000","last_clean":"2024-09-10T18:38:37.644520+0000","last_became_active":"2024-09-10T18:38:32.251960+0000","last_became_peered":"2024-09-10T18:38:32.251960+0000","last_unstale":"2024-09-10T18:38:37.644520+0000","last_undegraded":"2024-09-10T18:38:37.644520+0000","last_fullsized":"2024-09-10T18:38:37.644520+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:24.747601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.641898+0000","last_change":"2024-09-10T18:38:27.937582+0000","last_active":"2024-09-10T18:38:37.641898+0000","last_peered":"2024-09-10T18:38:37.641898+0000","last_clean":"2024-09-10T18:38:37.641898+0000","last_became_active":"2024-09-10T18:38:27.937268+0000","last_became_peered":"2024-09-10T18:38:27.937268+0000","last_unstale":"2024-09-10T18:38:37.641898+0000","last_undegraded":"2024-09-10T18:38:37.641898+0000","last_fullsized":"2024-09-10T18:38:37.641898+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:26:03.469222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704529+0000","last_change":"2024-09-10T18:38:25.926901+0000","last_active":"2024-09-10T18:38:37.704529+0000","last_peered":"2024-09-10T18:38:37.704529+0000","last_clean":"2024-09-10T18:38:37.704529+0000","last_became_active":"2024-09-10T18:38:25.925436+0000","last_became_peered":"2024-09-10T18:38:25.925436+0000","last_unstale":"2024-09-10T18:38:37.704529+0000","last_undegraded":"2024-09-10T18:38:37.704529+0000","last_fullsized":"2024-09-10T18:38:37.704529+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:51:16.142397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.156878+0000","last_change":"2024-09-10T18:38:29.940012+0000","last_active":"2024-09-10T18:38:38.156878+0000","last_peered":"2024-09-10T18:38:38.156878+0000","last_clean":"2024-09-10T18:38:38.156878+0000","last_became_active":"2024-09-10T18:38:29.939680+0000","last_became_peered":"2024-09-10T18:38:29.939680+0000","last_unstale":"2024-09-10T18:38:38.156878+0000","last_undegraded":"2024-09-10T18:38:38.156878+0000","last_fullsized":"2024-09-10T18:38:38.156878+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:44.113143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.701931+0000","last_change":"2024-09-10T18:38:31.951741+0000","last_active":"2024-09-10T18:38:37.701931+0000","last_peered":"2024-09-10T18:38:37.701931+0000","last_clean":"2024-09-10T18:38:37.701931+0000","last_became_active":"2024-09-10T18:38:31.950607+0000","last_became_peered":"2024-09-10T18:38:31.950607+0000","last_unstale":"2024-09-10T18:38:37.701931+0000","last_undegraded":"2024-09-10T18:38:37.701931+0000","last_fullsized":"2024-09-10T18:38:37.701931+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:25.539726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.643403+0000","last_change":"2024-09-10T18:38:31.944905+0000","last_active":"2024-09-10T18:38:37.643403+0000","last_peered":"2024-09-10T18:38:37.643403+0000","last_clean":"2024-09-10T18:38:37.643403+0000","last_became_active":"2024-09-10T18:38:31.944646+0000","last_became_peered":"2024-09-10T18:38:31.944646+0000","last_unstale":"2024-09-10T18:38:37.643403+0000","last_undegraded":"2024-09-10T18:38:37.643403+0000","last_fullsized":"2024-09-10T18:38:37.643403+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:30.929108+0000","last_clean_scrub_stamp":"2024-09-10T18:38:30.929108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:18.817888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.642044+0000","last_change":"2024-09-10T18:38:25.927049+0000","last_active":"2024-09-10T18:38:37.642044+0000","last_peered":"2024-09-10T18:38:37.642044+0000","last_clean":"2024-09-10T18:38:37.642044+0000","last_became_active":"2024-09-10T18:38:25.926795+0000","last_became_peered":"2024-09-10T18:38:25.926795+0000","last_unstale":"2024-09-10T18:38:37.642044+0000","last_undegraded":"2024-09-10T18:38:37.642044+0000","last_fullsized":"2024-09-10T18:38:37.642044+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:24.900625+0000","last_clean_scrub_stamp":"2024-09-10T18:38:24.900625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:14.811208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:38.157948+0000","last_change":"2024-09-10T18:38:27.932946+0000","last_active":"2024-09-10T18:38:38.157948+0000","last_peered":"2024-09-10T18:38:38.157948+0000","last_clean":"2024-09-10T18:38:38.157948+0000","last_became_active":"2024-09-10T18:38:27.932116+0000","last_became_peered":"2024-09-10T18:38:27.932116+0000","last_unstale":"2024-09-10T18:38:38.157948+0000","last_undegraded":"2024-09-10T18:38:38.157948+0000","last_fullsized":"2024-09-10T18:38:38.157948+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:26.914729+0000","last_clean_scrub_stamp":"2024-09-10T18:38:26.914729+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:59:53.858520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:38:37.704332+0000","last_change":"2024-09-10T18:38:29.936826+0000","last_active":"2024-09-10T18:38:37.704332+0000","last_peered":"2024-09-10T18:38:37.704332+0000","last_clean":"2024-09-10T18:38:37.704332+0000","last_became_active":"2024-09-10T18:38:29.936587+0000","last_became_peered":"2024-09-10T18:38:29.936587+0000","last_unstale":"2024-09-10T18:38:37.704332+0000","last_undegraded":"2024-09-10T18:38:37.704332+0000","last_fullsized":"2024-09-10T18:38:37.704332+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:38:28.918037+0000","last_clean_scrub_stamp":"2024-09-10T18:38:28.918037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:04:23.724546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":81,"num_read_kb":76,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1212416,"data_stored":823032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.5},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.55500000000000005}]},{"osd":1,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.187,"5min":0.187,"15min":0.187},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.47899999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.441}]},{"osd":3,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.46400000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.34200000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.63,"5min":0.63,"15min":0.63},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.45000000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.40799999999999997}]}]},{"osd":6,"up_from":42,"seq":180388626459,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298512,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446640,"statfs":{"total":95995035648,"available":95689359360,"internally_reserved":0,"allocated":1187840,"data_stored":821666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47799999999999998,"15min":0.47199999999999998},"min":{"1min":0.28299999999999997,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63600000000000001,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64300000000000002,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.53600000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.497,"15min":0.498},"min":{"1min":0.33900000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.65000000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.499,"15min":0.497},"min":{"1min":0.33400000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48099999999999998,"15min":0.47799999999999998},"min":{"1min":0.36099999999999999,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.65500000000000003,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.70899999999999996,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.624}]},{"osd":4,"last update":"Tue Sep 10 18:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.38,"15min":0.374},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.40699999999999997,"15min":0.39900000000000002},"min":{"1min":0.29099999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:39:53 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40300000000000002,"15min":0.39500000000000002},"min":{"1min":0.28999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.41599999999999998,"15min":0.40799999999999997},"min":{"1min":0.313,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.497}]},{"osd":7,"last update":"Tue Sep 10 18:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.442,"15min":0.442},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.61399999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297932,"kb_used_data":576,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447220,"statfs":{"total":95995035648,"available":95689953280,"internally_reserved":0,"allocated":589824,"data_stored":231295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.503,"15min":0.499},"min":{"1min":0.39600000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.63600000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.35999999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.51400000000000001,"15min":0.50900000000000001},"min":{"1min":0.43099999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.64300000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.61499999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.5}]},{"osd":2,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.501,"15min":0.497},"min":{"1min":0.38500000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.63800000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.51500000000000001},"min":{"1min":0.40699999999999997,"5min":0.37,"15min":0.37},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.43099999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.436,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.44900000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.64400000000000002,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.52300000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.36499999999999999,"15min":0.35899999999999999},"min":{"1min":0.31,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.36799999999999999,"15min":0.35499999999999998},"min":{"1min":0.316,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.53600000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:39:50 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.40300000000000002},"min":{"1min":0.30599999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42399999999999999,"15min":0.41599999999999998},"min":{"1min":0.34100000000000003,"5min":0.249,"15min":0.249},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.55900000000000005}]}]},{"osd":4,"up_from":30,"seq":128849018917,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297964,"kb_used_data":608,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447188,"statfs":{"total":95995035648,"available":95689920512,"internally_reserved":0,"allocated":622592,"data_stored":233349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.47299999999999998,"15min":0.46999999999999997},"min":{"1min":0.34999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.624,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.46999999999999997,"15min":0.46800000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.46400000000000002,"15min":0.46100000000000002},"min":{"1min":0.35199999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.63700000000000001,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.623,"5min":0.63,"15min":0.63},"last":0.55500000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.52100000000000002,"15min":0.51600000000000001},"min":{"1min":0.42399999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.50700000000000001,"15min":0.504},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.57199999999999995}]},{"osd":3,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.505,"15min":0.503},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67900000000000005,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.621}]},{"osd":5,"last update":"Tue Sep 10 18:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.35999999999999999},"min":{"1min":0.28299999999999997,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.442,"5min":0.374,"15min":0.36199999999999999},"min":{"1min":0.29099999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Sep 10 18:39:51 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.40300000000000002,"15min":0.40000000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57899999999999996,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.46500000000000002,"15min":0.45700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.58799999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297972,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447180,"statfs":{"total":95995035648,"available":95689912320,"internally_reserved":0,"allocated":630784,"data_stored":233319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:42 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.34499999999999997,"15min":0.35899999999999999},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.44800000000000001,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.38300000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.33900000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:39:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34000000000000002,"5min":0.38300000000000001,"15min":0.39600000000000002},"min":{"1min":0.23699999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.5,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.33600000000000002,"5min":0.379,"15min":0.39500000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47899999999999998,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.28499999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:39:42 2024","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.437,"15min":0.45300000000000001},"min":{"1min":0.29499999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53100000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.40600000000000003,"15min":0.41899999999999998},"min":{"1min":0.28499999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.55300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:39:01 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.53500000000000003,"15min":0.54400000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.627,"5min":0.749,"15min":0.749},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56200000000000006},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.68600000000000005,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.66600000000000004}]},{"osd":5,"last update":"Tue Sep 10 18:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.35099999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.752,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:38:52 2024","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.58899999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.69199999999999995}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297936,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447216,"statfs":{"total":95995035648,"available":95689949184,"internally_reserved":0,"allocated":593920,"data_stored":231727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.44400000000000001,"5min":0.496,"15min":0.496},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.376,"15min":0.37},"min":{"1min":0.26000000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48299999999999998,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.23599999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:39:23 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23100000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52500000000000002,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.221},{"interface":"front","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.38600000000000001},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.52300000000000002,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.32400000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:39:49 2024","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53200000000000003,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.309},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.42799999999999999,"15min":0.436},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.54200000000000004,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29499999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.56000000000000005,"15min":0.56399999999999995},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.76500000000000001,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.755,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:39:33 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56299999999999994,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76900000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58899999999999997,"15min":0.59199999999999997},"min":{"1min":0.44,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.72699999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53200000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:39:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.60199999999999998,"15min":0.61399999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73399999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.72199999999999998,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.621}]},{"osd":7,"last update":"Tue Sep 10 18:39:15 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":233655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.34599999999999997,"15min":0.34300000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.5,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.35599999999999998,"15min":0.33500000000000002},"min":{"1min":0.222,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.58899999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.311}]},{"osd":2,"last update":"Tue Sep 10 18:39:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.38600000000000001,"15min":0.39000000000000001},"min":{"1min":0.22,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.52000000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.39300000000000002,"15min":0.39100000000000001},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.54300000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.28299999999999997}]},{"osd":3,"last update":"Tue Sep 10 18:39:46 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.436,"15min":0.436},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.54400000000000004,"5min":1.593,"15min":1.593},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.375,"5min":0.41399999999999998,"15min":0.41899999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"last":0.318}]},{"osd":4,"last update":"Tue Sep 10 18:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.52200000000000002,"15min":0.52800000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:39:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.53600000000000003,"15min":0.53500000000000003},"min":{"1min":0.40899999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54300000000000004,"15min":0.54000000000000004},"min":{"1min":0.39100000000000001,"5min":0.376,"15min":0.376},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.499}]},{"osd":6,"last update":"Tue Sep 10 18:39:50 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55300000000000005,"15min":0.55800000000000005},"min":{"1min":0.40699999999999997,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.68000000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46700000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.47899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298540,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446612,"statfs":{"total":95995035648,"available":95689330688,"internally_reserved":0,"allocated":1212416,"data_stored":823637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.32100000000000001,"15min":0.30299999999999999},"min":{"1min":0.23799999999999999,"5min":0.18099999999999999,"15min":0.18099999999999999},"max":{"1min":0.45800000000000002,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.35899999999999999,"15min":0.32200000000000001},"min":{"1min":0.251,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.59699999999999998,"5min":1.04,"15min":1.04},"last":0.33000000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:39:25 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.255},{"interface":"front","average":{"1min":0.376,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.34300000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:39:43 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.46100000000000002,"15min":0.48799999999999999},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.64400000000000002,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.43099999999999999,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.307}]},{"osd":4,"last update":"Tue Sep 10 18:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.59699999999999998,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.59299999999999997,"15min":0.59999999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.70899999999999996,"5min":1.099,"15min":1.099},"last":0.48799999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.51400000000000001,"15min":0.50600000000000001},"min":{"1min":0.38800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52600000000000002,"15min":0.52200000000000002},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.58699999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:39:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40200000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69099999999999995,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56599999999999995,"15min":0.56699999999999995},"min":{"1min":0.41899999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73299999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.52400000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.46899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:39:56.472 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:39:56.472 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:39:56.472 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:39:56.472 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:39:56.677 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 podman[67301]: 2024-09-10 18:39:56.368686339 +0000 UTC m=+2.095652860 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:39:56.677 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 podman[67301]: 2024-09-10 18:39:56.475478423 +0000 UTC m=+2.202444986 container create 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:39:56.750 INFO:teuthology.orchestra.run.smithi131.stderr:Inferring config /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:57.177 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 podman[67301]: 2024-09-10 18:39:56.727808282 +0000 UTC m=+2.454774806 container init 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 podman[67301]: 2024-09-10 18:39:56.73309067 +0000 UTC m=+2.460057193 container start 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.736Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.736Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.737Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.737Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.737Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.737Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:39:57.178 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:39:57.179 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:39:57.180 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:39:57.181 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b[67375]: ts=2024-09-10T18:39:56.738Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 bash[67301]: 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 2024-09-10T18:39:57.182 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:39:56 smithi175 systemd[1]: Started Ceph node-exporter.b for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='client.34652 v1:172.21.15.131:0/568184779' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:57.862 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:57 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='client.34652 v1:172.21.15.131:0/568184779' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:39:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='client.34652 v1:172.21.15.131:0/568184779' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:57 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.635+0000 7fb8706f2640 1 -- 172.21.15.131:0/4266477996 >> v1:172.21.15.131:6790/0 conn(0x7fb868108ea0 legacy=0x7fb86810b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.635+0000 7fb8706f2640 1 -- 172.21.15.131:0/4266477996 shutdown_connections 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.635+0000 7fb8706f2640 1 -- 172.21.15.131:0/4266477996 >> 172.21.15.131:0/4266477996 conn(0x7fb8680f8130 msgr2=0x7fb8680fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.635+0000 7fb8706f2640 1 -- 172.21.15.131:0/4266477996 shutdown_connections 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.635+0000 7fb8706f2640 1 -- 172.21.15.131:0/4266477996 wait complete. 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb8706f2640 1 Processor -- start 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb8706f2640 1 -- start start 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb8706f2640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb86811bec0 con 0x7fb868108ea0 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb8706f2640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb86811d0c0 con 0x7fb868105310 2024-09-10T18:39:58.639 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb8706f2640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb86811e2c0 con 0x7fb868102550 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb86ec8f640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7fb868108ea0 0x7fb86811a5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:45912/0 (socket says 172.21.15.131:45912) 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb86ec8f640 1 -- 172.21.15.131:0/1781167124 learned_addr learned my addr 172.21.15.131:0/1781167124 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb86e48e640 1 --1- 172.21.15.131:0/1781167124 >> v1:172.21.15.131:6790/0 conn(0x7fb868102550 0x7fb8680fe560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:43112/0 (socket says 172.21.15.131:43112) 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4221929539 0 0) 0x7fb86811bec0 con 0x7fb868108ea0 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8500034e0 con 0x7fb868108ea0 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701792878 0 0) 0x7fb86811e2c0 con 0x7fb868102550 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86811bec0 con 0x7fb868102550 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274407304 0 0) 0x7fb86811d0c0 con 0x7fb868105310 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb86811e2c0 con 0x7fb868105310 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1472788609 0 0) 0x7fb86811bec0 con 0x7fb868102550 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.636+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb86811d0c0 con 0x7fb868102550 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb860001760 con 0x7fb868102550 2024-09-10T18:39:58.640 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 884770306 0 0) 0x7fb86811e2c0 con 0x7fb868105310 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb86811bec0 con 0x7fb868105310 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2827146663 0 0) 0x7fb8500034e0 con 0x7fb868108ea0 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb86811e2c0 con 0x7fb868108ea0 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb85c001840 con 0x7fb868105310 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb858002da0 con 0x7fb868108ea0 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2750937309 0 0) 0x7fb86811d0c0 con 0x7fb868102550 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 >> v1:172.21.15.175:6789/0 conn(0x7fb868105310 legacy=0x7fb8680fec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 >> v1:172.21.15.131:6789/0 conn(0x7fb868108ea0 legacy=0x7fb86811a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb86811f4c0 con 0x7fb868102550 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86811e4b0 con 0x7fb868102550 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86811e980 con 0x7fb868102550 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fb860002240 con 0x7fb868102550 2024-09-10T18:39:58.641 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.637+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fb860005840 con 0x7fb868102550 2024-09-10T18:39:58.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.638+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb854005010 con 0x7fb868102550 2024-09-10T18:39:58.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.638+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fb860004fc0 con 0x7fb868102550 2024-09-10T18:39:58.642 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.639+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 6168+0+0 (unknown 2876547528 0 0) 0x7fb860089c30 con 0x7fb868102550 2024-09-10T18:39:58.644 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.643+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fb860056960 con 0x7fb868102550 2024-09-10T18:39:58.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.842+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb854005840 con 0x7fb868102550 2024-09-10T18:39:58.844 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.843+0000 7fb866ffd640 1 -- 172.21.15.131:0/1781167124 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7fb860056300 con 0x7fb868102550 2024-09-10T18:39:58.845 INFO:teuthology.orchestra.run.smithi131.stdout: 2024-09-10T18:39:58.845 INFO:teuthology.orchestra.run.smithi131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:39:58.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.845+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 >> v1:172.21.15.131:6800/336444945 conn(0x7fb850074300 legacy=0x7fb8500767b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:58.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.845+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 >> v1:172.21.15.131:6790/0 conn(0x7fb868102550 legacy=0x7fb8680fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:39:58.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.845+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 shutdown_connections 2024-09-10T18:39:58.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.845+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 >> 172.21.15.131:0/1781167124 conn(0x7fb8680f8130 msgr2=0x7fb86810c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:39:58.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.845+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 shutdown_connections 2024-09-10T18:39:58.847 INFO:teuthology.orchestra.run.smithi131.stderr:2024-09-10T18:39:58.845+0000 7fb8706f2640 1 -- 172.21.15.131:0/1781167124 wait complete. 2024-09-10T18:39:59.546 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:39:59.547 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:39:59.547 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T18:39:59.561 INFO:tasks.workunit:Pulling workunits from ref fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:39:59.562 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T18:39:59.562 DEBUG:teuthology.orchestra.run.smithi131:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:39:59.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:39:59.578 INFO:teuthology.orchestra.run.smithi131.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T18:39:59.578 DEBUG:teuthology.orchestra.run.smithi131:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:39:59.634 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-10T18:39:59.634 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-10T18:39:59.689 INFO:tasks.workunit:timeout=3h 2024-09-10T18:39:59.690 INFO:tasks.workunit:cleanup=True 2024-09-10T18:39:59.690 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/adk3798/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:39:59.746 INFO:tasks.workunit.client.0.smithi131.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T18:39:59.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:59 smithi131 ceph-mon[29369]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:59.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:39:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1781167124' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:39:59.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:59 smithi131 ceph-mon[40885]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:59.809 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:39:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1781167124' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:39:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:59 smithi175 ceph-mon[33399]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:39:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:39:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1781167124' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:40:00.896 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:00 smithi131 ceph-mon[29369]: overall HEALTH_OK 2024-09-10T18:40:00.897 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:00 smithi131 ceph-mon[40885]: overall HEALTH_OK 2024-09-10T18:40:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:00 smithi175 ceph-mon[33399]: overall HEALTH_OK 2024-09-10T18:40:01.749 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:01 smithi131 ceph-mon[29369]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:01.750 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:01 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:01.750 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:01 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:01.750 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:01 smithi131 ceph-mon[40885]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:01.750 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:01 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:01.750 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:01 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:01 smithi175 ceph-mon[33399]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:01 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:01 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: Reconfiguring daemon alertmanager.a on smithi131 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:40:03.014 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: Reconfiguring daemon alertmanager.a on smithi131 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:40:03.015 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:40:03.016 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:03.016 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:03.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.232 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: Reconfiguring daemon alertmanager.a on smithi131 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]: dispatch 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:03.233 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:03.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:40:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[29369]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-10T18:40:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:40:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[40885]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:40:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:03 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.4", "id": [1, 5]}]': finished 2024-09-10T18:40:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-09-10T18:40:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-09-10T18:40:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:03 smithi175 ceph-mon[33399]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:40:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:03 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:04 smithi131 ceph-mon[29369]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:40:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:04 smithi131 ceph-mon[29369]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:04 smithi131 ceph-mon[40885]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:40:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:04 smithi131 ceph-mon[40885]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:04 smithi175 ceph-mon[33399]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:40:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:04 smithi175 ceph-mon[33399]: pgmap v36: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:40:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:05.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:05 smithi131 systemd[1]: Stopping Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:40:06.200 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:05 smithi131 ceph-mon[29369]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:40:06.200 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:05 smithi131 ceph-mon[40885]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:40:06.201 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:05 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[75820]: ts=2024-09-10T18:40:05.874Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:40:06.201 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:05 smithi131 podman[78742]: 2024-09-10 18:40:05.905261957 +0000 UTC m=+0.152518227 container died b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:06.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:05 smithi175 ceph-mon[33399]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-10T18:40:06.559 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 podman[78742]: 2024-09-10 18:40:06.306779551 +0000 UTC m=+0.554035820 container cleanup b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:06.559 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 bash[78742]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:40:06.811 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 podman[78760]: 2024-09-10 18:40:06.557948376 +0000 UTC m=+0.655439330 container remove b8b1e5d0c39673ee71c03c86704079a12032d6b06641ac2b8ff96df19bf16db8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:06.811 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 podman[78760]: 2024-09-10 18:40:06.650411564 +0000 UTC m=+0.747902517 volume remove 2a5612eb58b7e8107467975abbd6ddb7e0c9c6354d9e9656274b7259e9732732 2024-09-10T18:40:06.811 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:40:06.811 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 systemd[1]: Stopped Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:06 smithi131 ceph-mon[29369]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:06 smithi131 ceph-mon[40885]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:07.077 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:06 smithi131 systemd[1]: Starting Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:40:07.078 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 podman[78857]: 2024-09-10 18:40:06.933658932 +0000 UTC m=+0.019859385 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:40:07.078 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 podman[78857]: 2024-09-10 18:40:07.034764766 +0000 UTC m=+0.120965223 volume create 712669c0bf1c661cc410b985302a601fe0a777728a8b2cd8cb4f668891bec6d8 2024-09-10T18:40:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:06 smithi175 ceph-mon[33399]: pgmap v37: 132 pgs: 3 peering, 129 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:07.522 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 podman[78857]: 2024-09-10 18:40:07.177033439 +0000 UTC m=+0.263233892 container create 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:07.522 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 podman[78857]: 2024-09-10 18:40:07.521680087 +0000 UTC m=+0.607880548 container init 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:07.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 podman[78857]: 2024-09-10 18:40:07.527308659 +0000 UTC m=+0.613509113 container start 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:07.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.540Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:40:07.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.540Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:40:07.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.541Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.131 port=9094 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.542Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.570Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.570Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.573Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:07.573Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 bash[78857]: 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 2024-09-10T18:40:07.828 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:07 smithi131 systemd[1]: Started Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:08 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:08 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:08 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:08 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:09.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:08 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:09.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:08 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[29369]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:40:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[29369]: Reconfiguring daemon prometheus.a on smithi175 2024-09-10T18:40:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[29369]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 2 objects/s recovering 2024-09-10T18:40:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[29369]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T18:40:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[29369]: Cluster is now healthy 2024-09-10T18:40:09.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:09.543Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000977264s 2024-09-10T18:40:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[40885]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:40:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[40885]: Reconfiguring daemon prometheus.a on smithi175 2024-09-10T18:40:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[40885]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 2 objects/s recovering 2024-09-10T18:40:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[40885]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T18:40:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:09 smithi131 ceph-mon[40885]: Cluster is now healthy 2024-09-10T18:40:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:09 smithi175 ceph-mon[33399]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:40:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:09 smithi175 ceph-mon[33399]: Reconfiguring daemon prometheus.a on smithi175 2024-09-10T18:40:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:09 smithi175 ceph-mon[33399]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 2 objects/s recovering 2024-09-10T18:40:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:09 smithi175 ceph-mon[33399]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-10T18:40:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:09 smithi175 ceph-mon[33399]: Cluster is now healthy 2024-09-10T18:40:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:10 smithi175 ceph-mon[33399]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 2 objects/s recovering 2024-09-10T18:40:11.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:10 smithi131 ceph-mon[29369]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 2 objects/s recovering 2024-09-10T18:40:11.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:10 smithi131 ceph-mon[40885]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 44 B/s, 2 objects/s recovering 2024-09-10T18:40:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:13 smithi131 ceph-mon[29369]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 38 B/s, 2 objects/s recovering 2024-09-10T18:40:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:13 smithi131 ceph-mon[40885]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 38 B/s, 2 objects/s recovering 2024-09-10T18:40:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:13 smithi175 ceph-mon[33399]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 38 B/s, 2 objects/s recovering 2024-09-10T18:40:13.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:40:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:40:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:15.078 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:14 smithi175 systemd[1]: Stopping Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:40:15.406 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-mon[33399]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 399 B/s wr, 8 op/s; 34 B/s, 2 objects/s recovering 2024-09-10T18:40:15.407 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:15.407 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:40:15.407 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:40:15.407 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:40:15.407 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:40:15.407 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:40:15.407 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.193Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.194Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.194Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[65019]: ts=2024-09-10T18:40:15.194Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:40:15.408 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 podman[68305]: 2024-09-10 18:40:15.203821782 +0000 UTC m=+0.130028324 container died 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:15 smithi131 ceph-mon[29369]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 399 B/s wr, 8 op/s; 34 B/s, 2 objects/s recovering 2024-09-10T18:40:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:15 smithi131 ceph-mon[40885]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 399 B/s wr, 8 op/s; 34 B/s, 2 objects/s recovering 2024-09-10T18:40:15.677 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 podman[68305]: 2024-09-10 18:40:15.607821843 +0000 UTC m=+0.534028372 container cleanup 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:15.677 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 bash[68305]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:40:16.180 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 podman[68323]: 2024-09-10 18:40:15.825455412 +0000 UTC m=+0.619847353 container remove 9cd08576bac35a749728d5d92e2a292af559a2853c18c9b884927ad8bd93a03f (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:16.180 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:40:16.180 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 systemd[1]: Stopped Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:16.180 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.123s CPU time. 2024-09-10T18:40:16.181 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:15 smithi175 systemd[1]: Starting Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:40:16.462 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 podman[68431]: 2024-09-10 18:40:16.098545243 +0000 UTC m=+0.022017595 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:40:16.462 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 podman[68431]: 2024-09-10 18:40:16.234802211 +0000 UTC m=+0.158274564 container create 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:16.927 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 podman[68431]: 2024-09-10 18:40:16.46175992 +0000 UTC m=+0.385232274 container init 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:16.927 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 podman[68431]: 2024-09-10 18:40:16.466344073 +0000 UTC m=+0.389816427 container start 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:16.927 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.499Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.499Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.499Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi175 (none))" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.499Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.499Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.502Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.502Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.503Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.503Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.506Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.506Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.056µs 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.506Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:40:16.928 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.506Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.506Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.506Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.864µs wal_replay_duration=521.089µs wbl_replay_duration=138ns total_replay_duration=560.157µs 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.507Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.507Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.507Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.522Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.316044ms db_storage=757ns remote_storage=1.441µs web_handler=365ns query_engine=496ns scrape=392.092µs scrape_sd=31.728µs notify=19.377µs notify_sd=6.6µs rules=14.63157ms tracing=6.105µs 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.522Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:40:16.522Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 bash[68431]: 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 2024-09-10T18:40:16.929 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:40:16 smithi175 systemd[1]: Started Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:40:17.069 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:16 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:16] ENGINE Bus STOPPING 2024-09-10T18:40:17.069 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:40:17.069 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STOPPED 2024-09-10T18:40:17.069 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STARTING 2024-09-10T18:40:17.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Serving on http://:::9283 2024-09-10T18:40:17.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STARTED 2024-09-10T18:40:17.327 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STOPPING 2024-09-10T18:40:17.682 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:40:17.682 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STOPPED 2024-09-10T18:40:17.682 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STARTING 2024-09-10T18:40:17.683 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:40:17.547Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004729257s 2024-09-10T18:40:17.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s; 29 B/s, 1 objects/s recovering 2024-09-10T18:40:17.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:40:17.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:40:17.683 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:40:17.684 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:40:17.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:40:17.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:17.685 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:17.685 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s; 29 B/s, 1 objects/s recovering 2024-09-10T18:40:17.685 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.686 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:40:17.687 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:40:17.687 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:40:17.687 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:40:17.687 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.687 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:17.687 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 341 B/s wr, 6 op/s; 29 B/s, 1 objects/s recovering 2024-09-10T18:40:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:40:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi131.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi175.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi175.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:40:17.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:17.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:17.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:18.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Serving on http://:::9283 2024-09-10T18:40:18.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STARTED 2024-09-10T18:40:18.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:17 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:17] ENGINE Bus STOPPING 2024-09-10T18:40:18.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:40:18.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:18] ENGINE Bus STOPPED 2024-09-10T18:40:18.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:18] ENGINE Bus STARTING 2024-09-10T18:40:18.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:18] ENGINE Serving on http://:::9283 2024-09-10T18:40:18.577 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:40:18] ENGINE Bus STARTED 2024-09-10T18:40:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:19 smithi175 ceph-mon[33399]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s; 29 B/s, 1 objects/s recovering 2024-09-10T18:40:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:19 smithi131 ceph-mon[29369]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s; 29 B/s, 1 objects/s recovering 2024-09-10T18:40:20.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:19 smithi131 ceph-mon[40885]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.0 KiB/s rd, 341 B/s wr, 7 op/s; 29 B/s, 1 objects/s recovering 2024-09-10T18:40:20.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:20 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:20.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:20 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:20 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:20 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:20 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:20 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:21.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:21.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:21.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:21.925 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:21 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:21 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:22.977 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:22 smithi175 ceph-mon[33399]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:22 smithi131 ceph-mon[29369]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:23.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:22 smithi131 ceph-mon[40885]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:23.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:40:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:40:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:23 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:23 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:23] "GET /metrics HTTP/1.1" 200 219824 "" "Prometheus/2.43.0" 2024-09-10T18:40:24.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:23 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:25.055 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:24 smithi175 ceph-mon[33399]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:25.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:24 smithi131 ceph-mon[29369]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:25.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:24 smithi131 ceph-mon[40885]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:40:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:26.773 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:26 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:26.773 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:26 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:26 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:26 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:26.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:26 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:26.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:26 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:27 smithi131 ceph-mon[29369]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:27 smithi131 ceph-mon[40885]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:27 smithi175 ceph-mon[33399]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[29369]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[40885]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:29 smithi175 ceph-mon[33399]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:40:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:31 smithi131 ceph-mon[29369]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:31.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:31 smithi131 ceph-mon[40885]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:31 smithi175 ceph-mon[33399]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:33 smithi131 ceph-mon[29369]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:33 smithi131 ceph-mon[40885]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:33 smithi175 ceph-mon[33399]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:33.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:40:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:40:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:33] "GET /metrics HTTP/1.1" 200 219816 "" "Prometheus/2.43.0" 2024-09-10T18:40:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:35 smithi175 ceph-mon[33399]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:40:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:35 smithi131 ceph-mon[29369]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:35 smithi131 ceph-mon[40885]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:37 smithi131 ceph-mon[29369]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:37.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:37 smithi131 ceph-mon[40885]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:37 smithi175 ceph-mon[33399]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:39.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:39 smithi131 ceph-mon[29369]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:39.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:39 smithi131 ceph-mon[40885]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:39.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:39 smithi175 ceph-mon[33399]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:41.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:41 smithi131 ceph-mon[29369]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:41.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:41 smithi131 ceph-mon[40885]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:41 smithi175 ceph-mon[33399]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:43 smithi131 ceph-mon[29369]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:43.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:43 smithi131 ceph-mon[40885]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:43.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:43 smithi175 ceph-mon[33399]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:43.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:40:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:40:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:43] "GET /metrics HTTP/1.1" 200 219816 "" "Prometheus/2.43.0" 2024-09-10T18:40:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:44.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:45 smithi175 ceph-mon[33399]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:40:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:45 smithi131 ceph-mon[29369]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:45.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:45 smithi131 ceph-mon[40885]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:40:45.862 INFO:tasks.workunit.client.0.smithi131.stderr:Note: switching to 'fad5356a94e6044ce1062d748daa5a0f2ed53a7d'. 2024-09-10T18:40:45.862 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.862 INFO:tasks.workunit.client.0.smithi131.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: git switch -c 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr:Or undo this operation with: 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: git switch - 2024-09-10T18:40:45.863 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.864 INFO:tasks.workunit.client.0.smithi131.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T18:40:45.864 INFO:tasks.workunit.client.0.smithi131.stderr: 2024-09-10T18:40:45.864 INFO:tasks.workunit.client.0.smithi131.stderr:HEAD is now at fad5356a94e qa/upgrade: change pacific/octopus upgrade tests to use published image 2024-09-10T18:40:45.867 DEBUG:teuthology.orchestra.run.smithi131:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T18:40:45.931 INFO:tasks.workunit.client.0.smithi131.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T18:40:45.933 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:40:45.933 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T18:40:46.709 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T18:40:46.784 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T18:40:46.823 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:40:46.824 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:40:46.825 INFO:tasks.workunit.client.0.smithi131.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T18:40:46.862 INFO:tasks.workunit.client.0.smithi131.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:40:46.866 DEBUG:teuthology.orchestra.run.smithi131:> set -ex 2024-09-10T18:40:46.866 DEBUG:teuthology.orchestra.run.smithi131:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T18:40:46.922 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-10T18:40:46.924 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-10T18:40:46.924 DEBUG:teuthology.orchestra.run.smithi131:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-10T18:40:47.000 INFO:tasks.workunit.client.0.smithi131.stderr:+ parallel=1 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' '' = --serial ']' 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ color= 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -t 1 ']' 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ declare -A pids 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_aio 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_aio' 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_aio 2024-09-10T18:40:47.001 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.002 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_aio 2024-09-10T18:40:47.002 INFO:tasks.workunit.client.0.smithi131.stdout:test api_aio on pid 80114 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80114 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_aio on pid 80114' 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80114 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_aio_pp 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_aio_pp' 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_aio_pp 2024-09-10T18:40:47.005 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.006 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_aio_pp 2024-09-10T18:40:47.007 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80121 2024-09-10T18:40:47.008 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_aio_pp on pid 80121' 2024-09-10T18:40:47.008 INFO:tasks.workunit.client.0.smithi131.stdout:test api_aio_pp on pid 80121 2024-09-10T18:40:47.011 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80121 2024-09-10T18:40:47.011 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_io 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_io' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_io 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.012 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.013 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.014 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_io 2024-09-10T18:40:47.015 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80130 2024-09-10T18:40:47.016 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_io on pid 80130' 2024-09-10T18:40:47.016 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80130 2024-09-10T18:40:47.016 INFO:tasks.workunit.client.0.smithi131.stdout:test api_io on pid 80130 2024-09-10T18:40:47.020 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.020 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.020 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-10T18:40:47.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_io_pp 2024-09-10T18:40:47.020 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_io_pp' 2024-09-10T18:40:47.020 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_io_pp 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.021 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.022 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.023 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.024 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_io_pp 2024-09-10T18:40:47.025 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80141 2024-09-10T18:40:47.025 INFO:tasks.workunit.client.0.smithi131.stdout:test api_io_pp on pid 80141 2024-09-10T18:40:47.029 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_io_pp on pid 80141' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80141 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_asio 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_asio' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_asio 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.030 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.031 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.032 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.033 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.034 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.035 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_asio 2024-09-10T18:40:47.036 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80155 2024-09-10T18:40:47.037 INFO:tasks.workunit.client.0.smithi131.stdout:test api_asio on pid 80155 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_asio on pid 80155' 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80155 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_list 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.043 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_list' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_list 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.044 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.045 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.046 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.047 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.048 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.049 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.050 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80114/exe 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_list 2024-09-10T18:40:47.051 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80172 2024-09-10T18:40:47.052 INFO:tasks.workunit.client.0.smithi131.stdout:test api_list on pid 80172 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_list on pid 80172' 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80172 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_lock 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.055 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_lock' 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_lock 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.056 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.057 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.058 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.059 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.060 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.061 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.062 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_aio: /' 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_aio 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.063 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.064 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_lock 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.065 INFO:tasks.workunit.client.0.smithi131.stdout:test api_lock on pid 80193 2024-09-10T18:40:47.068 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80193 2024-09-10T18:40:47.068 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_lock on pid 80193' 2024-09-10T18:40:47.068 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80193 2024-09-10T18:40:47.068 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.068 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.068 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80121/exe 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_lock_pp 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_lock_pp' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_lock_pp 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.069 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.070 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.071 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.072 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.073 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.074 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.075 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_aio_pp 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.076 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.077 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.078 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.078 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80130/exe 2024-09-10T18:40:47.078 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_lock_pp 2024-09-10T18:40:47.078 INFO:tasks.workunit.client.0.smithi131.stdout:test api_lock_pp on pid 80213 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80213 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_lock_pp on pid 80213' 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80213 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.082 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.083 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.084 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.085 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.086 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80141/exe 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_misc 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_io: /' 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.087 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_misc' 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.088 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.089 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_io 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_io.log 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.090 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.091 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_misc 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_io_pp 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.092 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.093 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.094 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.094 INFO:tasks.workunit.client.0.smithi131.stdout:test api_misc on pid 80239 2024-09-10T18:40:47.099 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_misc 2024-09-10T18:40:47.099 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80239 2024-09-10T18:40:47.099 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_misc on pid 80239' 2024-09-10T18:40:47.099 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80239 2024-09-10T18:40:47.099 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_misc_pp 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.100 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_misc_pp' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.101 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.102 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.103 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_misc_pp 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.104 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.105 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.106 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80155/exe 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.107 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.108 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.109 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_misc_pp 2024-09-10T18:40:47.110 INFO:tasks.workunit.client.0.smithi131.stdout:test api_misc_pp on pid 80258 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80258 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_misc_pp on pid 80258' 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80258 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.112 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_tier_pp 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.113 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_tier_pp' 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_tier_pp 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.114 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.115 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.116 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80172/exe 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.117 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_asio: /' 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_tier_pp 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.118 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.119 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.119 INFO:tasks.workunit.client.0.smithi131.stdout:test api_tier_pp on pid 80276 2024-09-10T18:40:47.127 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80276 2024-09-10T18:40:47.127 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_tier_pp on pid 80276' 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80276 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_list: /' 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.128 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_list 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_pool 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.129 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.130 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.130 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.131 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.131 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_pool' 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.132 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_asio 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.133 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80193/exe 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.134 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.135 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_list.log 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.136 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_lock: /' 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.137 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.138 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_lock 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.139 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_pool 2024-09-10T18:40:47.140 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.141 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80213/exe 2024-09-10T18:40:47.141 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.141 INFO:tasks.workunit.client.0.smithi131.stdout:test api_pool on pid 80307 2024-09-10T18:40:47.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.145 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_pool 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80307 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_pool on pid 80307' 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80307 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.146 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_lock_pp 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_snapshots 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_snapshots' 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.147 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.148 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.149 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.150 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.151 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.152 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_snapshots 2024-09-10T18:40:47.153 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.154 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.155 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.155 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.155 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.155 INFO:tasks.workunit.client.0.smithi131.stdout:test api_snapshots on pid 80373 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_snapshots 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80373 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_snapshots on pid 80373' 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80373 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.160 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_snapshots_pp 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.161 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_snapshots_pp' 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.162 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.163 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80239/exe 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.164 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.165 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.166 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_misc 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.167 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.168 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.169 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.170 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_misc: /' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_snapshots_pp 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.171 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.172 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.173 INFO:tasks.workunit.client.0.smithi131.stdout:test api_snapshots_pp on pid 80414 2024-09-10T18:40:47.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_snapshots_pp 2024-09-10T18:40:47.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80414 2024-09-10T18:40:47.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_snapshots_pp on pid 80414' 2024-09-10T18:40:47.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80414 2024-09-10T18:40:47.177 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.177 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.178 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.179 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.180 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_stat 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_stat' 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.181 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.182 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.183 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80276/exe 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80258/exe 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.184 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.185 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.186 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.187 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.188 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_tier_pp 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_stat 2024-09-10T18:40:47.189 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-10T18:40:47.190 INFO:tasks.workunit.client.0.smithi131.stdout:test api_stat on pid 80466 2024-09-10T18:40:47.192 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_stat 2024-09-10T18:40:47.192 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80466 2024-09-10T18:40:47.192 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_stat on pid 80466' 2024-09-10T18:40:47.192 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80466 2024-09-10T18:40:47.192 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.193 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_misc_pp 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.194 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80307/exe 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_stat_pp 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_stat_pp' 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.195 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.196 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.197 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_pool 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_pool: /' 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.198 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.199 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.200 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_stat_pp 2024-09-10T18:40:47.200 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.200 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80373/exe 2024-09-10T18:40:47.200 INFO:tasks.workunit.client.0.smithi131.stdout:test api_stat_pp on pid 80506 2024-09-10T18:40:47.204 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_stat_pp 2024-09-10T18:40:47.204 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80506 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_stat_pp on pid 80506' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80506 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_watch_notify 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_watch_notify' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.205 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.206 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_watch_notify 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_snapshots 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.207 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.208 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.209 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.210 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.211 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.212 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.213 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.214 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.215 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.216 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80414/exe 2024-09-10T18:40:47.217 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.218 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_watch_notify 2024-09-10T18:40:47.218 INFO:tasks.workunit.client.0.smithi131.stdout:test api_watch_notify on pid 80549 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80549 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_watch_notify on pid 80549' 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80549 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.221 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_watch_notify_pp 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_watch_notify_pp' 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.222 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_snapshots_pp 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.223 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_watch_notify_pp 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.224 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.225 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.225 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.225 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.225 INFO:tasks.workunit.client.0.smithi131.stdout:test api_watch_notify_pp on pid 80562 2024-09-10T18:40:47.227 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_watch_notify_pp 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80562 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_watch_notify_pp on pid 80562' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80562 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_cmd 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_cmd' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.228 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.229 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_cmd 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.230 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.231 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.232 INFO:tasks.workunit.client.0.smithi131.stdout:test api_cmd on pid 80593 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_cmd 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80593 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_cmd on pid 80593' 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80593 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80466/exe 2024-09-10T18:40:47.236 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_cmd_pp 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_cmd_pp' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.237 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.238 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_stat 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.239 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_stat: /' 2024-09-10T18:40:47.240 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.241 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_cmd_pp 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.242 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-10T18:40:47.243 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80506/exe 2024-09-10T18:40:47.244 INFO:tasks.workunit.client.0.smithi131.stdout:test api_cmd_pp on pid 80649 2024-09-10T18:40:47.249 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_cmd_pp 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80649 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_cmd_pp on pid 80649' 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80649 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_service 2024-09-10T18:40:47.250 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_service' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.251 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.252 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.253 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.254 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.255 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.256 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.257 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_stat_pp 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.258 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_service 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.259 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-10T18:40:47.260 INFO:tasks.workunit.client.0.smithi131.stdout:test api_service on pid 80690 2024-09-10T18:40:47.260 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.260 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.260 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_service 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80690 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_service on pid 80690' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80690 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-10T18:40:47.261 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_service_pp 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_service_pp' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_service_pp 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.262 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.263 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.263 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.263 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.263 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.263 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.263 INFO:tasks.workunit.client.0.smithi131.stdout:test api_service_pp on pid 80702 2024-09-10T18:40:47.270 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.270 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.270 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.270 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.270 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.270 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.271 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_service_pp 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80702 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_service_pp on pid 80702' 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80702 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.272 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.273 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.274 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.275 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.276 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.277 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.278 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.279 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.280 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.281 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.282 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_c_write_operations 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.283 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_c_write_operations' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.284 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_c_write_operations 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.285 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.286 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80649/exe 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.287 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.288 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.289 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.290 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.291 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.292 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.292 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.292 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_c_write_operations 2024-09-10T18:40:47.292 INFO:tasks.workunit.client.0.smithi131.stdout:test api_c_write_operations on pid 80749 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80749 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_c_write_operations on pid 80749' 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80749 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-10T18:40:47.295 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80562/exe 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.296 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_c_read_operations 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_c_read_operations' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.297 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_c_read_operations 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.298 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_cmd_pp 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.299 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.300 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.301 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.301 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80593/exe 2024-09-10T18:40:47.301 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.301 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.301 INFO:tasks.workunit.client.0.smithi131.stdout:test api_c_read_operations on pid 80773 2024-09-10T18:40:47.303 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_c_read_operations 2024-09-10T18:40:47.303 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80773 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_c_read_operations on pid 80773' 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80773 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.304 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.305 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.306 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_cmd: /' 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.307 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s api_cls_remote_reads 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.308 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.309 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' api_cls_remote_reads' 2024-09-10T18:40:47.310 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.311 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80690/exe 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo api_cls_remote_reads 2024-09-10T18:40:47.312 INFO:tasks.workunit.client.0.smithi131.stdout:test api_cls_remote_reads on pid 80806 2024-09-10T18:40:47.318 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.318 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=api_cls_remote_reads 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80806 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test api_cls_remote_reads on pid 80806' 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80806 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.319 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.320 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_cmd 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s list_parallel 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_service 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_service.log 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.321 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.322 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.323 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.324 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_service: /' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80549/exe 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' list_parallel' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.325 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80702/exe 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.326 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.327 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.328 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.329 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_service_pp 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo list_parallel 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_watch_notify 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.330 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.331 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.332 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.333 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80749/exe 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.334 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_c_write_operations 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-10T18:40:47.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=list_parallel 2024-09-10T18:40:47.337 INFO:tasks.workunit.client.0.smithi131.stdout:test list_parallel on pid 80865 2024-09-10T18:40:47.337 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80865 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test list_parallel on pid 80865' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80865 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s open_pools_parallel 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.338 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' open_pools_parallel' 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.339 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo open_pools_parallel 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.340 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.341 INFO:tasks.workunit.client.0.smithi131.stdout:test open_pools_parallel on pid 80892 2024-09-10T18:40:47.343 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=open_pools_parallel 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80892 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test open_pools_parallel on pid 80892' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80892 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.344 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.345 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ printf %25s delete_pools_parallel 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:+ r=' delete_pools_parallel' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.346 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.347 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++ echo delete_pools_parallel 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.348 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.349 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.350 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ awk '{print $1}' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.351 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.352 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.353 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.354 INFO:tasks.workunit.client.0.smithi131.stderr:+ ff=delete_pools_parallel 2024-09-10T18:40:47.355 INFO:tasks.workunit.client.0.smithi131.stdout:test delete_pools_parallel on pid 80924 2024-09-10T18:40:47.364 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80924 2024-09-10T18:40:47.364 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo 'test delete_pools_parallel on pid 80924' 2024-09-10T18:40:47.364 INFO:tasks.workunit.client.0.smithi131.stderr:+ pids[$f]=80924 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+ ret=0 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80276 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80276 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.365 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.366 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.367 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.368 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:+ . /etc/bashrc 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -z '' ']' 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:++ shopt -q login_shell 2024-09-10T18:40:47.369 INFO:tasks.workunit.client.0.smithi131.stderr:+++ umask 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ SHELL=/bin/bash 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.370 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.371 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.372 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80892/exe 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.373 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.374 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:+++ return 2024-09-10T18:40:47.375 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export S_COLORS=auto 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:+++ S_COLORS=auto 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.376 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_open_pools_parallel 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.377 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80773/exe 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.378 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.379 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.380 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_c_read_operations 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:40:47.381 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.382 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/locale 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80806/exe 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.383 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.384 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-09-10T18:40:47.385 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80924/exe 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.386 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_delete_pools_parallel 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.387 INFO:tasks.workunit.client.0.smithi131.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:+++ unset LANG_backup config 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n '' ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ '[' '' ']' 2024-09-10T18:40:47.388 INFO:tasks.workunit.client.0.smithi131.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+++++ readlink /proc/80865/exe 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:++++ basename /usr/bin/bash 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_declare='declare -f' 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+++ which_opt=-f 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export which_declare 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+++ export -f which 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset i 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:++ unset -f pathmunge 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ export PATH 2024-09-10T18:40:47.389 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:40:47.390 INFO:tasks.workunit.client.0.smithi131.stderr:+ unset rc 2024-09-10T18:40:47.390 INFO:tasks.workunit.client.0.smithi131.stderr:+ sed 's/^/ list_parallel: /' 2024-09-10T18:40:47.390 INFO:tasks.workunit.client.0.smithi131.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-10T18:40:47.390 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph_test_rados_list_parallel 2024-09-10T18:40:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:47 smithi175 ceph-mon[33399]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:47 smithi131 ceph-mon[29369]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:47 smithi131 ceph-mon[40885]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/17374250' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:40:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:51 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:51 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/17374250' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/17374250' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:51 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/466278204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: pgmap v59: 196 pgs: 38 creating+peering, 26 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: pgmap v60: 196 pgs: 38 creating+peering, 26 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3100799302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-80207-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:52 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/466278204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: pgmap v59: 196 pgs: 38 creating+peering, 26 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: pgmap v60: 196 pgs: 38 creating+peering, 26 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3100799302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-80207-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/466278204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: pgmap v59: 196 pgs: 38 creating+peering, 26 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: pgmap v60: 196 pgs: 38 creating+peering, 26 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3100799302' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-80207-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: from='client.24688 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:53.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:52 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:53.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:40:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:53] "GET /metrics HTTP/1.1" 200 222637 "" "Prometheus/2.43.0" 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2746937371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2331518109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1237158012' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1802345484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1241313353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2137007801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1615280388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3616670043' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2482674750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1354144029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2646554710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: pgmap v62: 740 pgs: 38 creating+peering, 570 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1161458003' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.34733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.34748 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.34823 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.34853 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:54.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3100799302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-80207-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2478511371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2010962635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3167450673' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2003804548' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24832 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24850 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.24802 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2746937371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2331518109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1237158012' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1802345484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1241313353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2137007801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1615280388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3616670043' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2482674750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1354144029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2646554710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: pgmap v62: 740 pgs: 38 creating+peering, 570 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1161458003' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.34733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.34748 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.34823 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.34853 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3100799302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-80207-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2478511371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2010962635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3167450673' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2003804548' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24832 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.086 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24850 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.086 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.24802 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.086 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:53 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2746937371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2331518109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1237158012' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1802345484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1241313353' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2137007801' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1615280388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3616670043' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2482674750' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1354144029' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2646554710' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: pgmap v62: 740 pgs: 38 creating+peering, 570 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1161458003' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.34733 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.34748 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24760 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.34823 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24694 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.34853 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24712 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24742 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3100799302' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi131-80207-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24688 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi131-80182-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:54.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2478511371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2010962635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3167450673' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2003804548' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24796 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24832 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24850 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.24802 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:54.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:53 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] Global test environment set-up. 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (3 ms) 2024-09-10T18:40:54.183 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (3 ms) 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (4 ms) 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (1 ms) 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (0 ms) 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-10T18:40:54.184 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] 12 tests from AsioRados (25 ms total) 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [----------] Global test environment tear-down 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (7084 ms total) 2024-09-10T18:40:54.185 INFO:tasks.workunit.client.0.smithi131.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] Global test environment set-up. 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjects (330 ms) 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (34 ms) 2024-09-10T18:40:54.451 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo2,foo3 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo2 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo3 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo4,foo5 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo4 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo5 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo6,foo7 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo7 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo6 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-10T18:40:54.452 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo4 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo5 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo7 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo6 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo1 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo1 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo2 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo3 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (74 ms) 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 1 0 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 10 0 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 13 0 2024-09-10T18:40:54.453 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 7 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 14 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 0 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 15 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 11 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 5 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 8 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 6 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 3 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 4 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 12 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 9 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2 0 2024-09-10T18:40:54.454 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (63 ms) 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: x cursor=MIN 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=1 cursor=15:02547ec2:::1:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=10 cursor=15:52ea6a34:::10:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=13 cursor=15:566253c9:::13:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=7 cursor=15:5c6b0b28:::7:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=14 cursor=15:62a1935d:::14:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=0 cursor=15:6cac518f:::0:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=15 cursor=15:863748b0:::15:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=11 cursor=15:89d3ae78:::11:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=5 cursor=15:b29083e3:::5:head 2024-09-10T18:40:54.455 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=8 cursor=15:bd63b0f1:::8:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=6 cursor=15:c4fdafeb:::6:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=3 cursor=15:cfc208b3:::3:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=4 cursor=15:d83876eb:::4:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=12 cursor=15:de5d7c5f:::12:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=9 cursor=15:e960b815:::9:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > oid=2 cursor=15:f905c69b:::2:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=1 cursor=15:02547ec2:::1:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:02547ec2:::1:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:02547ec2:::1:head -> 1 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=10 cursor=15:52ea6a34:::10:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:52ea6a34:::10:head 2024-09-10T18:40:54.456 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:52ea6a34:::10:head -> 10 2024-09-10T18:40:54.457 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=13 cursor=15:566253c9:::13:head 2024-09-10T18:40:54.457 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:566253c9:::13:head 2024-09-10T18:40:54.457 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:566253c9:::13:head -> 13 2024-09-10T18:40:54.457 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=7 cursor=15:5c6b0b28:::7:head 2024-09-10T18:40:54.457 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:5c6b0b28:::7:head 2024-09-10T18:40:54.457 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:5c6b0b28:::7:head -> 7 2024-09-10T18:40:54.509 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=14 cursor=15:62a1935d:::14:head 2024-09-10T18:40:54.509 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:62a1935d:::14:head 2024-09-10T18:40:54.509 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:62a1935d:::14:head -> 14 2024-09-10T18:40:54.509 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=0 cursor=15:6cac518f:::0:head 2024-09-10T18:40:54.509 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:6cac518f:::0:head 2024-09-10T18:40:54.509 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:6cac518f:::0:head -> 0 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=15 cursor=15:863748b0:::15:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:863748b0:::15:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:863748b0:::15:head -> 15 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=11 cursor=15:89d3ae78:::11:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:89d3ae78:::11:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:89d3ae78:::11:head -> 11 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=5 cursor=15:b29083e3:::5:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:b29083e3:::5:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:b29083e3:::5:head -> 5 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=8 cursor=15:bd63b0f1:::8:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:bd63b0f1:::8:head 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:bd63b0f1:::8:head -> 8 2024-09-10T18:40:54.510 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=6 cursor=15:c4fdafeb:::6:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:c4fdafeb:::6:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:c4fdafeb:::6:head -> 6 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=3 cursor=15:cfc208b3:::3:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:cfc208b3:::3:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:cfc208b3:::3:head -> 3 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=4 cursor=15:d83876eb:::4:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:d83876eb:::4:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:d83876eb:::4:head -> 4 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=12 cursor=15:de5d7c5f:::12:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:de5d7c5f:::12:head 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:de5d7c5f:::12:head -> 12 2024-09-10T18:40:54.511 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=9 cursor=15:e960b815:::9:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:e960b815:::9:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:e960b815:::9:head -> 9 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : oid=2 cursor=15:f905c69b:::2:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:f905c69b:::2:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:f905c69b:::2:head -> 2 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:de5d7c5f:::12:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:de5d7c5f:::12:head expected=15:de5d7c5f:::12:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:de5d7c5f:::12:head -> 12 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=12 expected=12 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:d83876eb:::4:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:d83876eb:::4:head expected=15:d83876eb:::4:head 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:d83876eb:::4:head -> 4 2024-09-10T18:40:54.512 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=4 expected=4 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:cfc208b3:::3:head 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:cfc208b3:::3:head expected=15:cfc208b3:::3:head 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:cfc208b3:::3:head -> 3 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=3 expected=3 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:c4fdafeb:::6:head 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:c4fdafeb:::6:head expected=15:c4fdafeb:::6:head 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:c4fdafeb:::6:head -> 6 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=6 expected=6 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:bd63b0f1:::8:head 2024-09-10T18:40:54.513 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:bd63b0f1:::8:head expected=15:bd63b0f1:::8:head 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:bd63b0f1:::8:head -> 8 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=8 expected=8 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:b29083e3:::5:head 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:b29083e3:::5:head expected=15:b29083e3:::5:head 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:b29083e3:::5:head -> 5 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=5 expected=5 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:52ea6a34:::10:head 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:52ea6a34:::10:head expected=15:52ea6a34:::10:head 2024-09-10T18:40:54.514 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:52ea6a34:::10:head -> 10 2024-09-10T18:40:54.515 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=10 expected=10 2024-09-10T18:40:54.515 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:89d3ae78:::11:head 2024-09-10T18:40:54.515 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:89d3ae78:::11:head expected=15:89d3ae78:::11:head 2024-09-10T18:40:54.515 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:89d3ae78:::11:head -> 11 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : ent list_parallel: process_1_[81080]: starting. 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[81080]: creating pool ceph_test_rados_list_parallel.smithi131-81054 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[81080]: created object 0... 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[81080]: created object 25... 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[81080]: created object 49... 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[81080]: finishing. 2024-09-10T18:40:54.655 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_1_[81080]: shutting down. 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[81081]: starting. 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[81081]: listing objects. 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[81081]: listed object 0... 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[81081]: listed object 25... 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[81081]: saw 50 objects 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_2_[81081]: shutting down. 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: starting. 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: creating pool ceph_test_rados_list_parallel.smithi131-81054 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: created object 0... 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: created object 25... 2024-09-10T18:40:54.656 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: created object 49... 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: finishing. 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_3_[81704]: shutting down. 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[81705]: starting. 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[81705]: listing objects. 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[81705]: listed object 0... 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[81705]: listed object 25... 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[81705]: saw 43 objects 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_4_[81705]: shutting down. 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[81706]: starting. 2024-09-10T18:40:54.657 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[81706]: removed 25 objects... 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[81706]: removed half of the objects 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[81706]: removed 50 objects... 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[81706]: removed 50 objects 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_5_[81706]: shutting down. 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: starting. 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: creating pool ceph_test_rados_list_parallel.smithi131-81054 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: created object 0... 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: created object 25... 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: created object 49... 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: finishing. 2024-09-10T18:40:54.658 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_6_[81766]: shutting down. 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: starting. 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: listing objects. 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: listed object 0... 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: listed object 25... 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: listed object 50... 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: saw 56 objects 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_7_[81767]: shutting down. 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.659 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[81769]: starting. 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[81769]: added 25 objects... 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[81769]: added half of the objects 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[81769]: added 50 objects... 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[81769]: added 50 objects 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_8_[81769]: shutting down. 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.660 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: starting. 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: creating pool ceph_test_rados_list_parallel.smithi131-81054 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: created object 0... 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: created object 25... 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: created object 49... 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: finishing. 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_9_[81817]: shutting down. 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.833 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: starting. 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: listing objects. 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: listed object 0... 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: listed object 25... 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: listed object 50... 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: listed object 75... 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: saw 96 objects 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_10_[81818]: shutting down. 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.834 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[81821]: starting. 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[81821]: removed 25 objects... 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[81821]: removed half of the objects 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[81821]: removed 50 objects... 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[81821]: removed 50 objects 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_13_[81821]: shutting down. 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[81819]: starting. 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[81819]: added 25 objects... 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[81819]: added half of the objects 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[81819]: added 50 objects... 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[81819]: added 50 objects 2024-09-10T18:40:54.835 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_11_[81819]: shutting down. 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[81820]: starting. 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[81820]: added 25 objects... 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[81820]: added half of the objects 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[81820]: added 50 objects... 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[81820]: added 50 objects 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_12_[81820]: shutting down. 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.836 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: starting. 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: creating pool ceph_test_rados_list_parallel.smithi131-81054 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: created object 0... 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: created object 25... 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: created object 49... 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: finishing. 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_14_[81897]: shutting down. 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: starting. 2024-09-10T18:40:54.837 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listing objects. 2024-09-10T18:40:54.838 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listed object 0... 2024-09-10T18:40:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:40:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.34733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.34748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24850 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: pgmap v65: 868 pgs: 704 creating+peering, 32 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3455269250' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:55.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listed object 2 api_cmd_pp: Running main() from gmock_main.cc 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (49 ms) 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (44 ms) 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (8139 ms) 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (8232 ms total) 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: 2024-09-10T18:40:55.434 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-10T18:40:55.435 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (8232 ms total) 2024-09-10T18:40:55.435 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (8130 ms) 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (8130 ms total) 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (8130 ms total) 2024-09-10T18:40:55.441 INFO:tasks.workunit.client.0.smithi131.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-09-10T18:40:55.472 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (71 ms) 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (49 ms) 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (8099 ms) 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867384+0000 mon.a [INF] from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.473 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867462+0000 mon.a [INF] from='client.34733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867520+0000 mon.a [INF] from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867578+0000 mon.a [INF] from='client.34748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867631+0000 mon.a [INF] from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867691+0000 mon.a [INF] from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867767+0000 mon.a [INF] from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867821+0000 mon.a [INF] from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867885+0000 mon.a [INF] from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.867948+0000 mon.a [INF] from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868080+0000 mon.a [INF] from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.474 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868180+0000 mon.a [INF] from='client.24712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.34733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.34748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi131-80311-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.34733 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi131-80512-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi131-80551-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.34748 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi131-80676-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24760 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi131-80611-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.34823 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24694 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi131-80234-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.34853 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80776-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi131-80278-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24712 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi131-80288-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24850 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: pgmap v65: 868 pgs: 704 creating+peering, 32 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3455269250' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24850 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:40:55.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:55.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:55.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: pgmap v65: 868 pgs: 704 creating+peering, 32 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:55.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3455269250' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:55.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:56.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868270+0000 mon.a [INF] from='client.24742 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRado api_c_read_operations: Running main() from gmock_main.cc 2024-09-10T18:40:56.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (222 ms) 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (53 ms) 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (10 ms) 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (36 ms) 2024-09-10T18:40:56.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (6 ms) 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (8 ms) 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (7 ms) 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (3 ms) 2024-09-10T18:40:56.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (14 ms) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (12 ms) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (7 ms) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (398 ms total) 2024-09-10T18:40:56.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: 2024-09-10T18:40:56.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-10T18:40:56.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (9003 ms total) 2024-09-10T18:40:56.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout:5... 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listed object 50... 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listed object 75... 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listed object 100... 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: listed object 125... 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: saw 150 objects 2024-09-10T18:40:56.296 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_15_[81898]: shutting down. 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[81899]: starting. 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[81899]: added 25 objects... 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[81899]: added half of the objects 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[81899]: added 50 objects... 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[81899]: added 50 objects 2024-09-10T18:40:56.297 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: process_16_[81899]: shutting down. 2024-09-10T18:40:56.298 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.298 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.298 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.298 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.298 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******************************* 2024-09-10T18:40:56.298 INFO:tasks.workunit.client.0.smithi131.stdout: list_parallel: ******* SUCCESS ********** 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout:sTierPP_smithi131-80444-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868442+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/2305470951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi131-80220-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868526+0000 mon.a [INF] from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868632+0000 mon.a [INF] from='client.24796 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868720+0000 mon.a [INF] from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi131-80983-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868787+0000 mon.a [INF] from='client.24850 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-81033-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.868883+0000 mon.a [INF] from='client.24802 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi131-80834-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.872415+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:56.574 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:53.880510+0000 mon.a [INF] from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:56.575 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.429821+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/1172536137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.575 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.432727+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.575 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.435170+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.575 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.446338+0000 mon.a [INF] from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.575 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.453781+0000 mon.a [INF] from='client.24997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.590 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.454045+0000 mon.a [INF] from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: disp api_service: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:40:56.590 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] Global test environment set-up. 2024-09-10T18:40:56.590 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-10T18:40:56.590 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5031 ms) 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.RegisterLate (14 ms) 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: cluster: 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: id: 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: health: HEALTH_OK 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: services: 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mgr: y(active, since 110s), standbys: x 2024-09-10T18:40:56.591 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: data: 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pools: 6 pools, 132 pgs 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pgs: 132 active+clean 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: io: 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-10T18:40:56.592 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: cluster: 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: id: 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 19 pool(s) do not have an application enabled 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: services: 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mgr: y(active, since 112s), standbys: x 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-10T18:40:56.593 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: data: 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pools: 25 pools, 740 pgs 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pgs: 77.027% pgs unknown 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 5.135% pgs not active 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 570 unknown 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 132 active+clean 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 38 creating+peering 2024-09-10T18:40:56.594 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: io: 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: client: 639 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: cluster: 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: id: 0846fc60-6fa3-11ef-bcea-c7b262605968 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 8 pool(s) do not have an application enabled 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.595 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: services: 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: mgr: y(active, since 114s), standbys: x 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 3m) 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: data: 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: pools: 24 pools, 708 pgs 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-10T18:40:56.596 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:40:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1172536137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.24997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: onexx 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1172536137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.24997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: onexx 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:56.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:56.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:40:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1172536137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.24997 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: onexx 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:56.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:56.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: pgmap v67: 708 pgs: 416 creating+peering, 160 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:40:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.24997 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3359427156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:58.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3833955277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: pgmap v67: 708 pgs: 416 creating+peering, 160 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.24997 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3359427156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3833955277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:40:58.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:40:58.081 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: 12.3 deep-scrub starts 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: 12.3 deep-scrub ok 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25138 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.35123 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:40:58.083 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: 12.3 deep-scrub starts 2024-09-10T18:40:58.084 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: 12.3 deep-scrub ok 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25138 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.35123 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: twoxx 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/452508904' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:58.085 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: 12.8 deep-scrub starts 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: 12.8 deep-scrub ok 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: twoxx 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.086 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/452508904' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: 12.8 deep-scrub starts 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: 12.8 deep-scrub ok 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.087 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:57 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: pgmap v67: 708 pgs: 416 creating+peering, 160 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:40:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi131-80763-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.24997 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi131-80182-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3359427156' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3833955277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:40:58.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: 12.3 deep-scrub starts 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: 12.3 deep-scrub ok 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25138 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:40:58.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.35123 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: twoxx 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/452508904' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: 12.8 deep-scrub starts 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: 12.8 deep-scrub ok 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:57 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: open_pools_parallel: process_1_[81024]: starting. 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[81024]: creating pool ceph_test_rados_open_pools_parallel.smithi131-80971 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[81024]: created object 0... 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[81024]: created object 25... 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[81024]: created object 49... 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[81024]: finishing. 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_1_[81024]: shutting down. 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[81025]: starting. 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[81025]: rados_pool_create. 2024-09-10T18:40:58.303 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[81025]: rados_ioctx_create. 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_2_[81025]: shutting down. 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: starting. 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: creating pool ceph_test_rados_open_pools_parallel.smithi131-80971 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: created object 0... 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: created object 25... 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: created object 49... 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: finishing. 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_3_[81965]: shutting down. 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[81966]: starting. 2024-09-10T18:40:58.304 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[81966]: rados_pool_create. 2024-09-10T18:40:58.305 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[81966]: rados_ioctx_create. 2024-09-10T18:40:58.305 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: process_4_[81966]: shutting down. 2024-09-10T18:40:58.305 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-09-10T18:40:58.305 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******************************* 2024-09-10T18:40:58.305 INFO:tasks.workunit.client.0.smithi131.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-10T18:40:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25138 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25138 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:40:59.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.082 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:40:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:40:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25138 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:40:59.805 INFO:tasks.workunit.client.0.smithi131.stdout:atch 2024-09-10T18:40:59.805 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.468724+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:59.805 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.470339+0000 client.admin [INF] onexx 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.470671+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.470943+0000 mon.a [INF] from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.471540+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.473803+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.474338+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.476152+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.476325+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.476787+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.479305+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.483850+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.806 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.747904+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi131-80611-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.748225+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:55.749057+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.348500+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.353053+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.791385+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi131-80220-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.791491+0000 mon.a [INF] from='client.25138 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.791584+0000 mon.a [INF] from='client.35123 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.791952+0000 mon.a [INF] from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.792328+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.792649+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.793035+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.807 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.793238+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.795127+0000 mon.a [INF] from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.795805+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.803898+0000 mon.a [INF] from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.807382+0000 mon.a [INF] from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.807646+0000 mon.a [INF] from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.808130+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.808 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.808265+0000 mon.a [INF] from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.808538+0000 mon.a [INF] from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.849 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.808710+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.810239+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.810528+0000 mon.a [INF] from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.810783+0000 mon.a [INF] from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.811009+0000 mon.a [INF] from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.811391+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:57.811586+0000 mon.a [INF] from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.287791+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi131-80611-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.287911+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.288149+0000 mon.a [INF] from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.288300+0000 mon.a [INF] from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.294544+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.850 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.296919+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.297998+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.810158+0000 mon.a [INF] from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.810264+0000 mon.a [INF] from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.811059+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.811869+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.816932+0000 mon.a [INF] from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.822144+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.827127+0000 mon.a [INF] from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:58.827353+0000 mon.a [INF] from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.328367+0000 mon.a [INF] from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.328451+0000 mon.a [INF] from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:40:59.851 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.585002+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:40:59.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.585091+0000 mon.a [INF] from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:40:59.852 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.585142+0000 mon.a [INF] from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:41:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: 12.1 deep-scrub starts 2024-09-10T18:41:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: 12.1 deep-scrub ok 2024-09-10T18:41:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: pgmap v70: 684 pgs: 96 creating+peering, 588 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 516 op/s 2024-09-10T18:41:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: 12.9 deep-scrub starts 2024-09-10T18:41:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: 12.9 deep-scrub ok 2024-09-10T18:41:00.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:00.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.180 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.181 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/724762728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4186246833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.35189 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.35195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.182 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:40:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: 12.1 deep-scrub starts 2024-09-10T18:41:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: 12.1 deep-scrub ok 2024-09-10T18:41:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: pgmap v70: 684 pgs: 96 creating+peering, 588 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 516 op/s 2024-09-10T18:41:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: 12.9 deep-scrub starts 2024-09-10T18:41:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: 12.9 deep-scrub ok 2024-09-10T18:41:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:00.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: 12.1 deep-scrub starts 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: 12.1 deep-scrub ok 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: pgmap v70: 684 pgs: 96 creating+peering, 588 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 11 MiB/s rd, 34 MiB/s wr, 516 op/s 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: 12.9 deep-scrub starts 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: 12.9 deep-scrub ok 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi131-80676-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi131-80207-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/74604182' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi131-80288-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:00.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/724762728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4186246833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.35189 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.35195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.34988 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi131-80220-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi131-80234-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3075999025' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi131-80182-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:00.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.34988 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi131-80207-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/724762728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4186246833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.35189 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.35195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:00.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:40:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:00.347 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.585941+0000 mon.b [INF] from= delete_pools_parallel: process_1_[81068]: starting. 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[81068]: creating pool ceph_test_rados_delete_pools_parallel.smithi131-81048 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[81068]: created object 0... 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[81068]: created object 25... 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[81068]: created object 49... 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[81068]: finishing. 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_1_[81068]: shutting down. 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_2_[81069]: starting. 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_2_[81069]: deleting pool ceph_test_rados_delete_pools_parallel.smithi131-81048 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_2_[81069]: shutting down. 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: starting. 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: creating pool ceph_test_rados_delete_pools_parallel.smithi131-81048 2024-09-10T18:41:00.348 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: created object 0... 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: created object 25... 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: created object 49... 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: finishing. 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_3_[82093]: shutting down. 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[82095]: starting. 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[82095]: listing objects. 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[82095]: listed object 0... 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[82095]: listed object 25... 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[82095]: saw 50 objects 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_5_[82095]: shutting down. 2024-09-10T18:41:00.349 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:41:00.350 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_4_[82094]: starting. 2024-09-10T18:41:00.350 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_4_[82094]: deleting pool ceph_test_rados_delete_pools_parallel.smithi131-81048 2024-09-10T18:41:00.350 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: process_4_[82094]: shutting down. 2024-09-10T18:41:00.350 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:41:00.350 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:41:00.350 INFO:tasks.workunit.client.0.smithi131.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-10T18:41:00.427 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: Running main() from gmock_main.cc 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: seed 80234 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (275 ms) 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (8 ms) 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-10T18:41:00.428 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (6 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (5 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (5 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (4 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (6 ms) 2024-09-10T18:41:00.429 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (3 ms) 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (6 ms) 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (13 ms) 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-10T18:41:00.430 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (5 ms) 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (3 ms) 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (4 ms) 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (371 ms total) 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1071 ms) 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-10T18:41:00.431 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (11 ms) 2024-09-10T18:41:00.432 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-10T18:41:00.432 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-09-10T18:41:00.432 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-10T18:41:00.432 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (2 ms) 2024-09-10T18:41:00.432 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-10T18:41:00.432 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-09-10T18:41:01.098 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] 'client.? v1:172.21.15.131:0/4018858608' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:01.098 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.586241+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/724762728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.098 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.586878+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/4186246833' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.098 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.589838+0000 mon.a [INF] from='client.35189 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.098 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.590249+0000 mon.a [INF] from='client.35195 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.098 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.591446+0000 mon.a [INF] from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.591950+0000 mon.a [INF] from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.594956+0000 mon.a [INF] from='client.24775 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]: dispatch 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.804751+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.804910+0000 client.admin [INF] threexx 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:40:59.846363+0000 mon.a [INF] from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.336481+0000 mon.a [INF] from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.336646+0000 mon.a [INF] from='client.35189 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.336740+0000 mon.a [INF] from='client.35195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.602107+0000 mon.a [INF] from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:01.099 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.602262+0000 mon.a [INF] from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: 12.5 deep-scrub starts 2024-09-10T18:41:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: 12.5 deep-scrub ok 2024-09-10T18:41:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: 12.7 deep-scrub starts 2024-09-10T18:41:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: threexx 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: 12.7 deep-scrub ok 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.35189 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.35195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:01.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:01.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:01.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:01 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: 12.5 deep-scrub starts 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: 12.5 deep-scrub ok 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: 12.7 deep-scrub starts 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: threexx 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: 12.7 deep-scrub ok 2024-09-10T18:41:01.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.35189 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.35195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: 12.5 deep-scrub starts 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: 12.5 deep-scrub ok 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: 12.7 deep-scrub starts 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: threexx 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: 12.7 deep-scrub ok 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi131-80311-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:01.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.35189 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.35195 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.24775 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi131-80763-1"}]': finished 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:01.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:01 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:01.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.602810+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rado api_stat: Running main() from gmock_main.cc 2024-09-10T18:41:01.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-09-10T18:41:01.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] Global test environment set-up. 2024-09-10T18:41:01.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.Stat (243 ms) 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.StatNS (50 ms) 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (1 ms) 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3 ms) 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 4 tests from LibRadosStat (297 ms total) 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-10T18:41:01.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (559 ms) 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (6 ms) 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (200 ms) 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (10 ms) 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (775 ms total) 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [----------] Global test environment tear-down 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (14723 ms total) 2024-09-10T18:41:01.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat: [ PASSED ] 8 tests. 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: seed 80676 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (266 ms) 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (4 ms) 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-10T18:41:01.894 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (10 ms) 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (13 ms) 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (293 ms total) 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1088 ms) 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (0 ms) 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-10T18:41:01.895 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (6 ms) 2024-09-10T18:41:01.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1097 ms total) 2024-09-10T18:41:01.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: 2024-09-10T18:41:01.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-10T18:41:01.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (14716 ms total) 2024-09-10T18:41:01.896 INFO:tasks.workunit.client.0.smithi131.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-10T18:41:02.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: 12.4 deep-scrub starts 2024-09-10T18:41:02.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: pgmap v73: 644 pgs: 128 unknown, 96 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-10T18:41:02.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: 12.4 deep-scrub ok 2024-09-10T18:41:02.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: 12.0 deep-scrub starts 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: fourxx 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:41:02.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/273342333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: 12.4 deep-scrub starts 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: pgmap v73: 644 pgs: 128 unknown, 96 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: 12.4 deep-scrub ok 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: 12.0 deep-scrub starts 2024-09-10T18:41:02.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: fourxx 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:41:02.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/273342333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25237 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25237 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25237 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:41:02.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25237 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3623083616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2902932514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.585 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[29369]: from='client.35222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3623083616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2902932514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.586 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:02 smithi131 ceph-mon[40885]: from='client.35222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: 12.4 deep-scrub starts 2024-09-10T18:41:02.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: pgmap v73: 644 pgs: 128 unknown, 96 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 547 op/s 2024-09-10T18:41:02.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: 12.4 deep-scrub ok 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: 12.0 deep-scrub starts 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3885690283' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: fourxx 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi131-80288-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:02.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:02.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:41:02.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:02.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:41:02.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/273342333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25237 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi131-80611-6"}]': finished 2024-09-10T18:41:02.682 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25237 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi131-80182-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi131-80676-7"}]': finished 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1612818890' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/333999078' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3623083616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.683 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2902932514' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]: dispatch 2024-09-10T18:41:02.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]: dispatch 2024-09-10T18:41:02.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.25246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.684 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:02 smithi175 ceph-mon[33399]: from='client.35222 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:02.890 INFO:tasks.workunit.client.0.smithi131.stdout:LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-10T18:41:02.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (5 ms) 2024-09-10T18:41:02.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-10T18:41:02.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (2 ms) 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-10T18:41:02.891 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (2 ms) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (3 ms) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1144 ms total) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (15847 ms total) 2024-09-10T18:41:02.892 INFO:tasks.workunit.client.0.smithi131.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-09-10T18:41:02.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: Running main() from gmock_main.cc 2024-09-10T18:41:02.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-10T18:41:02.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] Global test environment set-up. 2024-09-10T18:41:02.898 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-10T18:41:02.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-10T18:41:02.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (267 ms) 2024-09-10T18:41:02.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-10T18:41:02.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-10T18:41:02.899 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-10T18:41:02.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-09-10T18:41:02.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-09-10T18:41:02.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-09-10T18:41:02.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-09-10T18:41:02.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: no timeout :/ 2024-09-10T18:41:02.900 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (42 ms) 2024-09-10T18:41:02.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-10T18:41:02.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (11 ms) 2024-09-10T18:41:02.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-10T18:41:02.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.Checksum (2 ms) 2024-09-10T18:41:02.901 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-10T18:41:02.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-09-10T18:41:02.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-10T18:41:02.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (6 ms) 2024-09-10T18:41:02.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-10T18:41:02.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-09-10T18:41:02.902 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-10T18:41:02.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-09-10T18:41:02.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-10T18:41:02.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-10T18:41:02.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-10T18:41:02.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-09-10T18:41:02.903 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-10T18:41:02.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-09-10T18:41:02.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-10T18:41:02.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.RmXattr (14 ms) 2024-09-10T18:41:02.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-10T18:41:02.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIo.XattrIter (5 ms) 2024-09-10T18:41:02.904 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 14 tests from LibRadosIo (372 ms total) 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1073 ms) 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (6 ms) 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (5 ms) 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-10T18:41:02.905 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (7 ms) 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-10T18:41:02.906 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-09-10T18:41:02.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1118 ms total) 2024-09-10T18:41:02.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: 2024-09-10T18:41:02.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [----------] Global test environment tear-down 2024-09-10T18:41:02.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [==========] 24 tests from 2 test suites ran. (15857 ms total) 2024-09-10T18:41:02.907 INFO:tasks.workunit.client.0.smithi131.stdout: api_io: [ PASSED ] 24 tests. 2024-09-10T18:41:03.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: 12.6 deep-scrub starts 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: 12.6 deep-scrub ok 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]: dispatch 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]: dispatch 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]: dispatch 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]: dispatch 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: pgmap v77: 660 pgs: 176 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:41:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.35222 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]': finished 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:03.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2"}]: dispatch 2024-09-10T18:41:03.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:03.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:03 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: 12.6 deep-scrub starts 2024-09-10T18:41:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: 12.6 deep-scrub ok 2024-09-10T18:41:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]: dispatch 2024-09-10T18:41:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]: dispatch 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]: dispatch 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]: dispatch 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: pgmap v77: 660 pgs: 176 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.35222 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]': finished 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]': finished 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2"}]: dispatch 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: 12.6 deep-scrub starts 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: 12.6 deep-scrub ok 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]: dispatch 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]: dispatch 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]: dispatch 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]: dispatch 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25126 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: pgmap v77: 660 pgs: 176 unknown, 64 creating+peering, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:03.808 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25180 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi131-80234-22"}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25147 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi131-80220-16"}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.35222 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pg_num": 4}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-4"}]': finished 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:41:03.809 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3270481321' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:03.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:03.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:03.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:03.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2"}]: dispatch 2024-09-10T18:41:03.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:03.810 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:41:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:03] "GET /metrics HTTP/1.1" 200 263445 "" "Prometheus/2.43.0" 2024-09-10T18:41:04.200 INFO:tasks.workunit.client.0.smithi131.stdout:s-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:04.200 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.603093+0000 mon.b [INF] from='client.? v1:172.21.15.131:0/912900929' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:04.200 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.603640+0000 mon.c [INF] from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.200 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.604787+0000 mon.a [INF] from='client.? v1:172.21.15.131:0/537294255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi131-80611-6"}]: dispatch 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.608623+0000 mon.a [INF] from='client.35201 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.611037+0000 mon.a [INF] from='client.25009 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-4", "overlaypool": "test-rados-api-smithi131-80444-4-cache"}]: dispatch 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: got: 2024-09-10T18:41:00.611213+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi131-80676-7"}]: dispatch 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (8766 ms) 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (16985 ms total) 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: 2024-09-10T18:41:04.201 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-10T18:41:04.202 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (16985 ms total) 2024-09-10T18:41:04.202 INFO:tasks.workunit.client.0.smithi131.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-10T18:41:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: 12.2 deep-scrub starts 2024-09-10T18:41:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: 12.2 deep-scrub ok 2024-09-10T18:41:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/936192628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.35228 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/889382230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[40885]: from='client.25264 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: 12.2 deep-scrub starts 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: 12.2 deep-scrub ok 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:04.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/936192628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.35228 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/889382230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:04.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:04 smithi131 ceph-mon[29369]: from='client.25264 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: 12.2 deep-scrub starts 2024-09-10T18:41:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: 12.2 deep-scrub ok 2024-09-10T18:41:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.25009 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-4", "tierpool": "test-rados-api-smithi131-80444-4-cache"}]': finished 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi131-80399-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/936192628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.35228 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:04.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:04.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/889382230' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:04.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:04.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:04 smithi175 ceph-mon[33399]: from='client.25264 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: pgmap v80: 592 pgs: 4 creating+peering, 64 unknown, 524 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.3 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-09-10T18:41:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: 12.0 deep-scrub starts 2024-09-10T18:41:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: 12.0 deep-scrub ok 2024-09-10T18:41:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35228 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]': finished 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.25264 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/329515189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3965796519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]: dispatch 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[29369]: from='client.35240 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: pgmap v80: 592 pgs: 4 creating+peering, 64 unknown, 524 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.3 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: 12.0 deep-scrub starts 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: 12.0 deep-scrub ok 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35228 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:05.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]': finished 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.25264 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/329515189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3965796519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]: dispatch 2024-09-10T18:41:05.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:05 smithi131 ceph-mon[40885]: from='client.35240 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-10T18:41:05.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:41:05.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-10T18:41:05.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-10T18:41:05.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: seed 80311 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (244 ms) 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (37 ms) 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1072 ms) 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1032 ms) 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (3 ms) 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-10T18:41:05.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (3 ms) 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (5 ms) 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2399 ms total) 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1053 ms) 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-10T18:41:05.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (6 ms) 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1003 ms) 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (3 ms) 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-09-10T18:41:05.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (3 ms) 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3080 ms total) 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (18830 ms total) 2024-09-10T18:41:05.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-10T18:41:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: pgmap v80: 592 pgs: 4 creating+peering, 64 unknown, 524 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.3 KiB/s rd, 7.5 KiB/s wr, 9 op/s 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: 12.0 deep-scrub starts 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: 12.0 deep-scrub ok 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35228 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi131-80512-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]': finished 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:41:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.25264 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi131-80182-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4008181820' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/329515189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3965796519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:05.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]: dispatch 2024-09-10T18:41:05.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:05.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:05 smithi175 ceph-mon[33399]: from='client.35240 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]': finished 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35240 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]': finished 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35240 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:06.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:06 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: Running main() from gmock_main.cc 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] Global test environment set-up. 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (275 ms) 2024-09-10T18:41:06.887 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockShared (9 ms) 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1055 ms) 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1038 ms) 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.Unlock (8 ms) 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-10T18:41:06.888 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (6 ms) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (3 ms) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLock (2398 ms total) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (585 ms) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (3 ms) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1009 ms) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1013 ms) 2024-09-10T18:41:06.889 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (2 ms) 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (2624 ms total) 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [----------] Global test environment tear-down 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (19820 ms total) 2024-09-10T18:41:06.890 INFO:tasks.workunit.client.0.smithi131.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-10T18:41:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35174 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi131-80311-10"}]': finished 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "mode": "writeback"}]': finished 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35240 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2606658533' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]: dispatch 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3555011504' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:41:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:06.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:06.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:06 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: pgmap v83: 520 pgs: 4 creating+peering, 96 unknown, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.7 KiB/s wr, 2 op/s 2024-09-10T18:41:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: pool 'PoolQuotaPP_smithi131-80207-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2078800594' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:08.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:07 smithi175 ceph-mon[33399]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:08.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: pgmap v83: 520 pgs: 4 creating+peering, 96 unknown, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.7 KiB/s wr, 2 op/s 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: pool 'PoolQuotaPP_smithi131-80207-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:08.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2078800594' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[29369]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: pgmap v83: 520 pgs: 4 creating+peering, 96 unknown, 420 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.7 KiB/s wr, 2 op/s 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: pool 'PoolQuotaPP_smithi131-80207-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.25201 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi131-80288-10"}]': finished 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.35201 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi131-80399-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:41:08.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:08.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:41:08.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:08.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:08.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2078800594' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:08.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:07 smithi131 ceph-mon[40885]: from='client.25291 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.019 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-10T18:41:09.019 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1100 ms) 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (5 ms) 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1105 ms total) 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: 2024-09-10T18:41:09.020 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (48 ms) 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4042 ms) 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (4 ms) 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (3 ms) 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193135600 notify_id 339302416385 notifier_gid 35210 2024-09-10T18:41:09.021 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193135600 notify_id 339302416386 notifier_gid 35210 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193138752 notify_id 339302416389 notifier_gid 35210 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (4 ms) 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193142112 notify_id 339302416386 notifier_gid 35210 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (6 ms) 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193162336 notify_id 339302416387 notifier_gid 35210 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (4 ms) 2024-09-10T18:41:09.022 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193165696 notify_id 339302416390 notifier_gid 35210 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: trying... 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193162336 notify_id 339302416387 notifier_gid 35210 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: timed out 2024-09-10T18:41:09.023 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: pgmap v86: 552 pgs: 64 creating+peering, 32 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3170963726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3962814123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.25300 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: pgmap v86: 552 pgs: 64 creating+peering, 32 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3170963726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3962814123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.25300 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.25291 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi131-80182-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: pgmap v86: 552 pgs: 64 creating+peering, 32 unknown, 456 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3170963726' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3962814123' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.25300 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:09.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[29369]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[29369]: from='client.25300 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[29369]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:41:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[40885]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[40885]: from='client.25300 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[40885]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:41:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:10 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi131-80834-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:10 smithi175 ceph-mon[33399]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:10 smithi175 ceph-mon[33399]: from='client.25300 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:10 smithi175 ceph-mon[33399]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:41:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:11.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:41:11.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/882613702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.27361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2638998553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.27313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: pgmap v89: 560 pgs: 48 creating+peering, 120 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-10T18:41:11.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:11.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.27361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: from='client.27313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.332 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[29369]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:41:11.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:41:11.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/882613702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.332 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.27361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2638998553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.27313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: pgmap v89: 560 pgs: 48 creating+peering, 120 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-10T18:41:11.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:11.333 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.334 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.27361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.334 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: from='client.27313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.334 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:11 smithi131 ceph-mon[40885]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:41:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:41:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/882613702' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.27361 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2638998553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.27313 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: pgmap v89: 560 pgs: 48 creating+peering, 120 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-10T18:41:11.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:11.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.27361 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi131-80182-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: from='client.27313 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi131-80551-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:11.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:11 smithi175 ceph-mon[33399]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:41:12.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch api_watch_notify: Running main() from gmock_main.cc 2024-09-10T18:41:12.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-10T18:41:12.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-10T18:41:12.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-10T18:41:12.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify_test_cb 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (269 ms) 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94366946101440 err -107 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1081 ms) 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94366946117088 err -107 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1025 ms) 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 292057776128 cookie 94366946138832 2024-09-10T18:41:12.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (5 ms) 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 292057776128 cookie 94366946160064 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (4 ms) 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 292057776129 cookie 94366946187392 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (5 ms) 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 292057776129 cookie 94366946160064 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 292057776129 cookie 94366946187392 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (6 ms) 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-10T18:41:12.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 296352743426 cookie 94366946160064 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 300647710723 cookie 94366946160064 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3119 ms) 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94366946160064 err -107 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_cb from 24802 notify_id 330712481793 cookie 94366946187392 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5006 ms) 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94366946187392 err -107 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1005 ms) 2024-09-10T18:41:12.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11525 ms total) 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: watch_notify_test_cb 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1137 ms) 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1137 ms total) 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (25690 ms total) 2024-09-10T18:41:12.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:41:12.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:12.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2288590941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:12.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:12.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:12.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:12.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:12 smithi175 ceph-mon[33399]: from='client.29710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:13.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:41:13.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:13.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2288590941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:13.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[40885]: from='client.29710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:41:13.093 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:13.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2288590941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2049771169' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:13.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]: dispatch 2024-09-10T18:41:13.094 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:12 smithi131 ceph-mon[29369]: from='client.29710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.606 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:41:13.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-10T18:41:13.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (8204 ms) 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (2854 ms) 2024-09-10T18:41:13.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3048 ms) 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2557 ms) 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (2994 ms) 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (4134 ms) 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (2891 ms) 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (26682 ms total) 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: 2024-09-10T18:41:13.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-10T18:41:13.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (26683 ms total) 2024-09-10T18:41:13.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: pgmap v92: 520 pgs: 21 creating+peering, 107 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]': finished 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.29710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:41:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3423143139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.30868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2553066534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.30814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:13.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:13.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:13 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: pgmap v92: 520 pgs: 21 creating+peering, 107 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:41:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]': finished 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.29710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3423143139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.30868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2553066534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.30814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:13] "GET /metrics HTTP/1.1" 200 263445 "" "Prometheus/2.43.0" 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: pgmap v92: 520 pgs: 21 creating+peering, 107 unknown, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-7"}]': finished 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.35255 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi131-80834-12"}]': finished 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.29710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80848-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:41:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3423143139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.30868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2553066534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.30814 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:14.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:14.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:13 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.30868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.30814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:41:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:14.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:14.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.30868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.30814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.30868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi131-80182-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:15.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.30814 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:15.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:15.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:41:15.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:15.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]: dispatch 2024-09-10T18:41:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: pgmap v95: 456 pgs: 64 creating+peering, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]': finished 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1446625840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.33082 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: pgmap v95: 456 pgs: 64 creating+peering, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-09-10T18:41:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]': finished 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1446625840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.33082 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: pgmap v95: 456 pgs: 64 creating+peering, 392 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2 op/s 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-7"}]': finished 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:16.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:41:16.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1446625840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.33082 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:16.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:16.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:16.637 INFO:tasks.workunit.client.0.smithi131.stdout: pgs: 22.599% pgs unknown 2024-09-10T18:41:16.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 58.757% pgs not active 2024-09-10T18:41:16.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 416 creating+peering 2024-09-10T18:41:16.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 160 unknown 2024-09-10T18:41:16.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 132 active+clean 2024-09-10T18:41:16.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:41:16.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: io: 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: client: 1.1 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4337 ms) 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ OK ] LibRadosService.Status (20023 ms) 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] 4 tests from LibRadosService (29405 ms total) 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [----------] Global test environment tear-down 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [==========] 4 tests from 1 test suite ran. (29405 ms total) 2024-09-10T18:41:16.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_service: [ PASSED ] 4 tests. 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: from='client.33082 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: pgmap v98: 456 pgs: 64 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2129174268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: from='client.34318 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:17.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:16 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: from='client.33082 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: pgmap v98: 456 pgs: 64 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2129174268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: from='client.34318 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:17.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: from='client.33082 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: pgmap v98: 456 pgs: 64 unknown, 32 creating+peering, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2129174268' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: from='client.34318 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:17.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:16 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:17.727 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:41:17.727 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-10T18:41:17.727 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-10T18:41:17.727 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-10T18:41:17.727 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5044 ms) 2024-09-10T18:41:17.727 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (40 ms) 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20031 ms) 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: attempt 0 of 20 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5370 ms) 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30485 ms total) 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30485 ms total) 2024-09-10T18:41:17.728 INFO:tasks.workunit.client.0.smithi131.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-10T18:41:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='client.34318 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:41:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1324548850' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='client.34318 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:18.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1324548850' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:41:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi131-80512-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='client.34318 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi131-80182-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:41:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1324548850' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:41:19.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:41:19.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3995799645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.36730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/942829435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.46052 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: pgmap v101: 464 pgs: 64 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]: dispatch 2024-09-10T18:41:19.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.36730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.46052 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]': finished 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3468421922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:41:19.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3995799645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.36730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/942829435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.46052 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: pgmap v101: 464 pgs: 64 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.36730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.46052 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.583 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:19.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]': finished 2024-09-10T18:41:19.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:41:19.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:19.584 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3468421922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:41:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:41:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3995799645' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.36730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/942829435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.46052 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:19.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:19.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]: dispatch 2024-09-10T18:41:19.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: pgmap v101: 464 pgs: 64 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:41:19.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]: dispatch 2024-09-10T18:41:19.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:19.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.36730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.46052 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi131-80207-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:19.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:19.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4"}]': finished 2024-09-10T18:41:19.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:41:19.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:19.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3468421922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.37906 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]': finished 2024-09-10T18:41:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.37906 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]': finished 2024-09-10T18:41:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:41:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:20.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.37906 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]': finished 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.37906 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]': finished 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.37906 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-9"}]': finished 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.37906 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi131-80182-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6"}]': finished 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2935117863' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:41:20.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]: dispatch 2024-09-10T18:41:20.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]: dispatch 2024-09-10T18:41:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: pgmap v104: 496 pgs: 64 unknown, 32 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-10T18:41:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:41:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]': finished 2024-09-10T18:41:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:41:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/654853294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.40261 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: pgmap v104: 496 pgs: 64 unknown, 32 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-10T18:41:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:41:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]': finished 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/654853294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.40261 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: pgmap v104: 496 pgs: 64 unknown, 32 creating+peering, 400 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.35210 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi131-80763-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:41:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-9", "mode": "writeback"}]': finished 2024-09-10T18:41:21.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:41:21.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/654853294' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:21.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.40261 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:21.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:21.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout:_notify_pp: flushed 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3003 ms) 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: trying... 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193162336 notify_id 352187318279 notifier_gid 35210 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: timed out 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushed 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3024 ms) 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-10T18:41:21.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: List watches 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193164464 notify_id 365072220168 notifier_gid 35210 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 done 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: watch_check 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: unwatch2 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: done 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3039 ms) 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: List watches 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: handle_notify cookie 94687193164464 notify_id 377957122057 notifier_gid 35210 2024-09-10T18:41:21.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: notify2 done 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: watch_check 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: unwatch2 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: flushing 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: done 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3034 ms) 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16226 ms total) 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (34734 ms total) 2024-09-10T18:41:21.938 INFO:tasks.workunit.client.0.smithi131.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-10T18:41:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.40261 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:41:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2634433490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.41443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:22.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:22.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:22.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:22.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.40261 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:22.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:22.830 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:41:22.831 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2634433490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:22.831 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.41443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:22.831 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:22.831 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.40261 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi131-80207-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2634433490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.41443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:22.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:22.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]: dispatch 2024-09-10T18:41:23.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:41:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: pgmap v107: 428 pgs: 64 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: from='client.41443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]': finished 2024-09-10T18:41:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:41:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: pgmap v107: 428 pgs: 64 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: from='client.41443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]': finished 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:23] "GET /metrics HTTP/1.1" 200 247150 "" "Prometheus/2.43.0" 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: pgmap v107: 428 pgs: 64 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: from='client.41443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi131-80182-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:23.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-9"}]': finished 2024-09-10T18:41:23.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:41:23.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:24 smithi175 ceph-mon[33399]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:41:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/758235934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:24 smithi175 ceph-mon[33399]: from='client.43756 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[29369]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/758235934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[29369]: from='client.43756 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80497-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[40885]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/758235934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:24 smithi131 ceph-mon[40885]: from='client.43756 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: pgmap v110: 396 pgs: 32 creating+peering, 32 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.43756 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:41:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4226146834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:25.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:25 smithi175 ceph-mon[33399]: from='client.45043 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: pgmap v110: 396 pgs: 32 creating+peering, 32 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.43756 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:41:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4226146834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[29369]: from='client.45043 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: pgmap v110: 396 pgs: 32 creating+peering, 32 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s wr, 0 op/s 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.43756 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi131-80207-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4226146834' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:25 smithi131 ceph-mon[40885]: from='client.45043 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:26.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-10T18:41:26.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] Global test environment set-up. 2024-09-10T18:41:26.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-10T18:41:26.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-10T18:41:26.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-10T18:41:26.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-10T18:41:26.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-09-10T18:41:26.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-10T18:41:26.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-10T18:41:26.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-10T18:41:26.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-09-10T18:40:47.167+0000 7f7ff0cb9600 -1 failed for service _ceph-mon._tcp 2024-09-10T18:41:26.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-09-10T18:40:47.167+0000 7f7ff0cb9600 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-10T18:41:26.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (61 ms) 2024-09-10T18:41:26.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-10T18:41:26.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5009 ms) 2024-09-10T18:41:26.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5070 ms total) 2024-09-10T18:41:26.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-09-10T18:41:26.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-10T18:41:26.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-10T18:41:26.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55cb675c22d0 2024-09-10T18:41:26.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 0x55cb675c6ad0 2024-09-10T18:41:26.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: started 2 aios 2024-09-10T18:41:26.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55cb675c22d0 2024-09-10T18:41:26.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: waiting 0x55cb675c6ad0 2024-09-10T18:41:26.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: done. 2024-09-10T18:41:26.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (7411 ms) 2024-09-10T18:41:26.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (7411 ms total) 2024-09-10T18:41:26.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-09-10T18:41:26.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-10T18:41:26.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-10T18:41:26.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-10T18:41:26.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-10T18:41:26.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.Exec (68 ms) 2024-09-10T18:41:26.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-10T18:41:26.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-09-10T18:41:26.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-10T18:41:26.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (2 ms) 2024-09-10T18:41:26.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-10T18:41:26.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5467 ms) 2024-09-10T18:41:26.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-10T18:41:26.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-10T18:41:26.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-10T18:41:26.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-10T18:41:26.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-10T18:41:26.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (19206 ms) 2024-09-10T18:41:26.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] 8 tests from LibRadosMisc (24748 ms total) 2024-09-10T18:41:26.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: 2024-09-10T18:41:26.920 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [----------] Global test environment tear-down 2024-09-10T18:41:26.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (39789 ms total) 2024-09-10T18:41:26.921 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-10T18:41:26.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:26.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:26 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:26.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:41:26.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:26 smithi175 ceph-mon[33399]: from='client.45043 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:26.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:26 smithi175 ceph-mon[33399]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:41:26.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80497-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[29369]: from='client.45043 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[29369]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80497-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:27.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:27.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:27.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:41:27.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[40885]: from='client.45043 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi131-80182-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:27.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[40885]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:41:27.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80497-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:27.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: pgmap v113: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-10T18:41:27.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:27.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:41:27.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:41:27.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:41:27.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:41:27.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:41:27.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:41:27.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:41:27.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:41:27.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:41:27.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:41:27.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: 12.8 deep-scrub starts 2024-09-10T18:41:27.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80497-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:27.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:41:27.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:27.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4154093305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:27.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:27.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache"}]: dispatch 2024-09-10T18:41:27.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.47713 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:27.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:27.931 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:27.932 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: 12.1 deep-scrub starts 2024-09-10T18:41:27.932 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: 12.1 deep-scrub ok 2024-09-10T18:41:27.932 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:27 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: pgmap v113: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-10T18:41:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: 12.8 deep-scrub starts 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80497-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:41:28.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4154093305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache"}]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.47713 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: 12.1 deep-scrub starts 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: 12.1 deep-scrub ok 2024-09-10T18:41:28.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: pgmap v113: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:41:28.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: 12.8 deep-scrub starts 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80497-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4154093305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache"}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.47713 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: 12.1 deep-scrub starts 2024-09-10T18:41:28.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: 12.1 deep-scrub ok 2024-09-10T18:41:28.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:27 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:41:28.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: 12.8 deep-scrub ok 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: 12.3 deep-scrub starts 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: 12.3 deep-scrub ok 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache"}]': finished 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.47713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:28.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1555598035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:28.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='client.47734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: 12.9 deep-scrub starts 2024-09-10T18:41:28.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: 12.9 deep-scrub ok 2024-09-10T18:41:28.930 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:28 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:28.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:41:28.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:41:28.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:41:28.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: 12.8 deep-scrub ok 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: 12.3 deep-scrub starts 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: 12.3 deep-scrub ok 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache"}]': finished 2024-09-10T18:41:28.979 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.47713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1555598035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.0"}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.1"}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.2"}]: dispatch 2024-09-10T18:41:28.980 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.3"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.4"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.5"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.6"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.7"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.8"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.131:0/1062418757' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "12.9"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: 12.8 deep-scrub ok 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: 12.3 deep-scrub starts 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: 12.3 deep-scrub ok 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:28.981 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4108998136' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80497-10", "tierpool":"test-rados-api-smithi131-80497-10-cache"}]': finished 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.47713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi131-80207-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3334570263' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]: dispatch 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1555598035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='client.47734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.982 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: 12.9 deep-scrub starts 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: 12.9 deep-scrub ok 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='client.47734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: 12.9 deep-scrub starts 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: 12.9 deep-scrub ok 2024-09-10T18:41:28.983 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:28 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:29.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: Running main() from gmock_main.cc 2024-09-10T18:41:29.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] Global test environment set-up. 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolList (8312 ms) 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3382 ms) 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (2530 ms) 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2554 ms) 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-10T18:41:29.917 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3030 ms) 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (4982 ms) 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5025 ms) 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4981 ms) 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (7998 ms) 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] 9 tests from LibRadosPools (42794 ms total) 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [----------] Global test environment tear-down 2024-09-10T18:41:29.918 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (42794 ms total) 2024-09-10T18:41:29.919 INFO:tasks.workunit.client.0.smithi131.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: pgmap v116: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]': finished 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.47734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: 12.5 deep-scrub starts 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: 12.5 deep-scrub ok 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:29.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:29 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: pgmap v116: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:41:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:41:30.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]': finished 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.47734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: 12.5 deep-scrub starts 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: 12.5 deep-scrub ok 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:30.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: pgmap v116: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 2 op/s 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.33142 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi131-80512-9"}]': finished 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-11"}]': finished 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.47734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi131-80182-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:41:30.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: 12.5 deep-scrub starts 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: 12.5 deep-scrub ok 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:30.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:29 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:30.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:30.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: 12.7 deep-scrub starts 2024-09-10T18:41:30.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: 12.7 deep-scrub ok 2024-09-10T18:41:30.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]': finished 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1053170373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:30.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:30 smithi175 ceph-mon[33399]: from='client.56561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: 12.7 deep-scrub starts 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: 12.7 deep-scrub ok 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]': finished 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:31.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1053170373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[29369]: from='client.56561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: 12.7 deep-scrub starts 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: 12.7 deep-scrub ok 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-11", "mode": "writeback"}]': finished 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:41:31.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1053170373' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:31.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:30 smithi131 ceph-mon[40885]: from='client.56561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: pgmap v119: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 1 op/s 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: 12.6 deep-scrub starts 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: 12.6 deep-scrub ok 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: from='client.56561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:41:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/187007781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-80182-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:31.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:31.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: pgmap v119: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 1 op/s 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: 12.6 deep-scrub starts 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: 12.6 deep-scrub ok 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: from='client.56561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/187007781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-80182-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:32.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: pgmap v119: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 1 op/s 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: 12.6 deep-scrub starts 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: 12.6 deep-scrub ok 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: from='client.56561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/187007781' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-80182-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:32.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/187007781' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-80182-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/187007781' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-80182-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi131-80512-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/187007781' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi131-80182-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]: dispatch 2024-09-10T18:41:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:33.637 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: pgmap v122: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: 12.2 deep-scrub starts 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: 12.2 deep-scrub ok 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]': finished 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3015533288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: from='client.56576 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:33.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:33 smithi175 ceph-mon[33399]: 12.4 deep-scrub starts 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:33] "GET /metrics HTTP/1.1" 200 240015 "" "Prometheus/2.43.0" 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: pgmap v122: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: 12.2 deep-scrub starts 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: 12.2 deep-scrub ok 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]': finished 2024-09-10T18:41:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3015533288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: from='client.56576 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[29369]: 12.4 deep-scrub starts 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: pgmap v122: 364 pgs: 40 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: 12.2 deep-scrub starts 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: 12.2 deep-scrub ok 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-11"}]': finished 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:41:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3015533288' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:34.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: from='client.56576 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:34.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:33 smithi131 ceph-mon[40885]: 12.4 deep-scrub starts 2024-09-10T18:41:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:34 smithi175 ceph-mon[33399]: 12.4 deep-scrub ok 2024-09-10T18:41:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:34 smithi175 ceph-mon[33399]: from='client.56576 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:34 smithi175 ceph-mon[33399]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:41:34.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1021120484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-80182-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[29369]: 12.4 deep-scrub ok 2024-09-10T18:41:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[29369]: from='client.56576 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[29369]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:41:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1021120484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-80182-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:35.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[40885]: 12.4 deep-scrub ok 2024-09-10T18:41:35.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:35.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:35.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[40885]: from='client.56576 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi131-80207-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:35.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[40885]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:41:35.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1021120484' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi131-80182-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: pgmap v125: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: 12.0 deep-scrub starts 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: 12.0 deep-scrub ok 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1021120484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi131-80182-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:41:35.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:35.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: pgmap v125: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: 12.0 deep-scrub starts 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: 12.0 deep-scrub ok 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1021120484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi131-80182-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: pgmap v125: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: 12.0 deep-scrub starts 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: 12.0 deep-scrub ok 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1021120484' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi131-80182-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:36.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:36 smithi175 ceph-mon[33399]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:41:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/639089044' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:36 smithi175 ceph-mon[33399]: from='client.47767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:37.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[29369]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:41:37.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/639089044' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[29369]: from='client.47767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:37.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:37.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[40885]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:41:37.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/639089044' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:36 smithi131 ceph-mon[40885]: from='client.47767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:37 smithi175 ceph-mon[33399]: pgmap v128: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-10T18:41:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:37 smithi175 ceph-mon[33399]: from='client.47767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:37 smithi175 ceph-mon[33399]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:41:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2630216226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-80182-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[29369]: pgmap v128: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[29369]: from='client.47767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[29369]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2630216226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-80182-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[40885]: pgmap v128: 364 pgs: 64 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-10T18:41:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[40885]: from='client.47767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi131-80207-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[40885]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:41:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2630216226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-80182-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2630216226' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-80182-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:38 smithi175 ceph-mon[33399]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:41:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2630216226' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-80182-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:38 smithi131 ceph-mon[29369]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:41:39.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:39.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2630216226' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi131-80182-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:39.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:38 smithi131 ceph-mon[40885]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:41:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:39 smithi175 ceph-mon[33399]: pgmap v131: 364 pgs: 364 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 81 KiB/s wr, 78 op/s 2024-09-10T18:41:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:39 smithi175 ceph-mon[33399]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:41:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1614224841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-80207-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[29369]: pgmap v131: 364 pgs: 364 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 81 KiB/s wr, 78 op/s 2024-09-10T18:41:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[29369]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:41:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1614224841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-80207-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:40.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[40885]: pgmap v131: 364 pgs: 364 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.5 KiB/s rd, 81 KiB/s wr, 78 op/s 2024-09-10T18:41:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[40885]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:41:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1614224841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-80207-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:41:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:41:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1614224841' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-80207-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3535771671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.47785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1614224841' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-80207-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3535771671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.47785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1614224841' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi131-80207-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:41.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3535771671' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.47785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:41.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:41:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: pgmap v134: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-09-10T18:41:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.47785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:41:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:41:41.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:41.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:41:41.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:41.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:41:41.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:41 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: pgmap v134: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-09-10T18:41:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.47785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:41:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:41:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:41:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:42.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: pgmap v134: 396 pgs: 64 unknown, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 5.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.47785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi131-80182-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:42.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:41:42.080 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:41 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (2419 ms) 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-10T18:41:42.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (1998 ms) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2313 ms) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1286 ms) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (8016 ms total) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4034 ms) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4042 ms) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (8076 ms total) 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-09-10T18:41:42.926 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2997 ms) 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1984 ms) 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1996 ms) 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2078 ms) 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9055 ms total) 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-10T18:41:42.927 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4047 ms) 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3980 ms) 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8027 ms total) 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (55796 ms total) 2024-09-10T18:41:42.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]': finished 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]: dispatch 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]: dispatch 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1843883177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-80207-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:43.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:43.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:42 smithi175 ceph-mon[33399]: pgmap v137: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]': finished 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]: dispatch 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]: dispatch 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1843883177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-80207-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[29369]: pgmap v137: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-13"}]': finished 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]: dispatch 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1618240593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]: dispatch 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]: dispatch 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1843883177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-80207-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:42 smithi131 ceph-mon[40885]: pgmap v137: 356 pgs: 32 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:43.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:41:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:43] "GET /metrics HTTP/1.1" 200 240015 "" "Prometheus/2.43.0" 2024-09-10T18:41:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]': finished 2024-09-10T18:41:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1843883177' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-80207-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3417415416' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.56615 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:41:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:41:44.079 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]': finished 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1843883177' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-80207-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3417415416' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.56615 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:44.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:41:44.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:41:44.082 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:43 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:41:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:41:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-13", "mode": "writeback"}]': finished 2024-09-10T18:41:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.47746 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi131-80512-14"}]': finished 2024-09-10T18:41:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1843883177' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi131-80207-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3417415416' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.56615 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:41:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:41:44.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:43 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:41:45.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:45.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: from='client.56615 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: 97.7 scrub starts 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: 97.7 scrub ok 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: 97.1 scrub starts 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: 97.1 scrub ok 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: pgmap v140: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 42 KiB/s wr, 88 op/s 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: 97.2 scrub starts 2024-09-10T18:41:45.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:44 smithi175 ceph-mon[33399]: 97.2 scrub ok 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: from='client.56615 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: 97.7 scrub starts 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: 97.7 scrub ok 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: 97.1 scrub starts 2024-09-10T18:41:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: 97.1 scrub ok 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: pgmap v140: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 42 KiB/s wr, 88 op/s 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: 97.2 scrub starts 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[29369]: 97.2 scrub ok 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: from='client.56615 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi131-80182-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: 97.7 scrub starts 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: 97.7 scrub ok 2024-09-10T18:41:45.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: 97.1 scrub starts 2024-09-10T18:41:45.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: 97.1 scrub ok 2024-09-10T18:41:45.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: pgmap v140: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 24 KiB/s rd, 42 KiB/s wr, 88 op/s 2024-09-10T18:41:45.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: 97.2 scrub starts 2024-09-10T18:41:45.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:44 smithi131 ceph-mon[40885]: 97.2 scrub ok 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2734830682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: from='client.56621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: 97.3 scrub starts 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: 97.3 scrub ok 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: 97.5 scrub starts 2024-09-10T18:41:46.300 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:45 smithi175 ceph-mon[33399]: 97.5 scrub ok 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2734830682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: from='client.56621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: 97.3 scrub starts 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: 97.3 scrub ok 2024-09-10T18:41:46.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: 97.5 scrub starts 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[29369]: 97.5 scrub ok 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2734830682' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: from='client.56621 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: 97.3 scrub starts 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: 97.3 scrub ok 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: 97.5 scrub starts 2024-09-10T18:41:46.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:45 smithi131 ceph-mon[40885]: 97.5 scrub ok 2024-09-10T18:41:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: 97.8 scrub starts 2024-09-10T18:41:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: 97.8 scrub ok 2024-09-10T18:41:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: from='client.56621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3796051603' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: pgmap v143: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 50 KiB/s wr, 90 op/s 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: 97.8 scrub starts 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: 97.8 scrub ok 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: from='client.56621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3796051603' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: pgmap v143: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 50 KiB/s wr, 90 op/s 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:41:47.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: 97.8 scrub starts 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: 97.8 scrub ok 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: from='client.56621 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi131-80207-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3796051603' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: from='client.56627 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:47.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: pgmap v143: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 28 KiB/s rd, 50 KiB/s wr, 90 op/s 2024-09-10T18:41:47.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:47.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: from='client.56627 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi131-80182-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:47.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:41:47.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[29369]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:41:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1171523894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[29369]: from='client.56633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[40885]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:41:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1171523894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:48 smithi131 ceph-mon[40885]: from='client.56633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:48.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:48 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:48.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:48 smithi175 ceph-mon[33399]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:41:48.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:48.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1171523894' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:48.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:48 smithi175 ceph-mon[33399]: from='client.56633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: 97.0 scrub starts 2024-09-10T18:41:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: 97.0 scrub ok 2024-09-10T18:41:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: pgmap v146: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 37 op/s 2024-09-10T18:41:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: 97.6 scrub starts 2024-09-10T18:41:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: 97.6 scrub ok 2024-09-10T18:41:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: from='client.56633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2955981848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[29369]: from='client.56639 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: 97.0 scrub starts 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: 97.0 scrub ok 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: pgmap v146: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 37 op/s 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: 97.6 scrub starts 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: 97.6 scrub ok 2024-09-10T18:41:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:49.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: from='client.56633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:49.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:41:49.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2955981848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:49.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:49 smithi131 ceph-mon[40885]: from='client.56639 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: 97.0 scrub starts 2024-09-10T18:41:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: 97.0 scrub ok 2024-09-10T18:41:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: pgmap v146: 356 pgs: 32 creating+peering, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 30 KiB/s rd, 37 op/s 2024-09-10T18:41:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: 97.6 scrub starts 2024-09-10T18:41:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: 97.6 scrub ok 2024-09-10T18:41:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:49.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: from='client.56633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi131-80207-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:49.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:41:49.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2955981848' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:49.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:49 smithi175 ceph-mon[33399]: from='client.56639 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: 97.9 scrub starts 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: 97.9 scrub ok 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: 97.4 scrub starts 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: 97.4 scrub ok 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: from='client.56639 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:41:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/491795789' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[29369]: from='client.56645 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: 97.9 scrub starts 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: 97.9 scrub ok 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: 97.4 scrub starts 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: 97.4 scrub ok 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: from='client.56639 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/491795789' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:50 smithi131 ceph-mon[40885]: from='client.56645 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: 97.9 scrub starts 2024-09-10T18:41:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: 97.9 scrub ok 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: 97.4 scrub starts 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: 97.4 scrub ok 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: from='client.56639 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80182-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/491795789' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:50.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:50 smithi175 ceph-mon[33399]: from='client.56645 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[29369]: pgmap v149: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 34 op/s 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[29369]: from='client.56645 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[29369]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1752632996' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[29369]: from='client.47830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:51.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[40885]: pgmap v149: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 34 op/s 2024-09-10T18:41:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[40885]: from='client.56645 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[40885]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:41:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1752632996' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:51 smithi131 ceph-mon[40885]: from='client.47830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:51 smithi175 ceph-mon[33399]: pgmap v149: 356 pgs: 64 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 29 KiB/s rd, 34 op/s 2024-09-10T18:41:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:51 smithi175 ceph-mon[33399]: from='client.56645 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80551-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:51 smithi175 ceph-mon[33399]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:41:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1752632996' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:51 smithi175 ceph-mon[33399]: from='client.47830 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[29369]: from='client.47830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[29369]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:41:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4081406497' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[29369]: from='client.47839 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[40885]: from='client.47830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[40885]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4081406497' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:52 smithi131 ceph-mon[40885]: from='client.47839 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:52 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:52 smithi175 ceph-mon[33399]: from='client.47830 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi131-80207-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:52 smithi175 ceph-mon[33399]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4081406497' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:52 smithi175 ceph-mon[33399]: from='client.47839 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[29369]: pgmap v152: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[29369]: from='client.47839 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[29369]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[40885]: pgmap v152: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[40885]: from='client.47839 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:53.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[40885]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:41:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:53 smithi175 ceph-mon[33399]: pgmap v152: 388 pgs: 96 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:41:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:53 smithi175 ceph-mon[33399]: from='client.47839 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi131-80182-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:53 smithi175 ceph-mon[33399]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:41:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:53.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:41:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:41:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:41:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:53] "GET /metrics HTTP/1.1" 200 237826 "" "Prometheus/2.43.0" 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2583871486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2583871486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:54.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:54 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2583871486' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:54.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:54.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:54 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:41:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:41:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: pgmap v155: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2583871486' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:41:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1944821389' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:55 smithi175 ceph-mon[33399]: from='client.47863 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: pgmap v155: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2583871486' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:41:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1944821389' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[29369]: from='client.47863 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: pgmap v155: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2583871486' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1944821389' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:55.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:41:55.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:55 smithi131 ceph-mon[40885]: from='client.47863 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:56.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:56.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:56 smithi175 ceph-mon[33399]: from='client.47863 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:56.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:56 smithi175 ceph-mon[33399]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:41:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:56 smithi131 ceph-mon[29369]: from='client.47863 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:56 smithi131 ceph-mon[29369]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:41:56.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:56.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:56 smithi131 ceph-mon[40885]: from='client.47863 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi131-80182-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:56.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:56 smithi131 ceph-mon[40885]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:41:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: pgmap v158: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2"}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1531145791' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[29369]: from='client.56702 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: pgmap v158: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:57.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:41:57.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2"}]: dispatch 2024-09-10T18:41:57.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1531145791' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:57 smithi131 ceph-mon[40885]: from='client.56702 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: pgmap v158: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:41:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi131-80551-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2"}]: dispatch 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1531145791' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:57.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:57 smithi175 ceph-mon[33399]: from='client.56702 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.56702 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3115973241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.56708 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.729 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.56702 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3115973241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.56708 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.730 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:58 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi131-80450-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.56702 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi131-80207-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3115973241' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.56708 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:58 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:41:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: pgmap v161: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:41:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: from='client.56708 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:41:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: pgmap v161: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: from='client.56708 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:41:59 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: pgmap v161: 364 pgs: 32 creating+peering, 332 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:41:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:41:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: from='client.56708 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi131-80182-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:41:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:41:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:41:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:41:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:41:59 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1348648529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:42:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1348648529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:42:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:00 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:42:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:42:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:42:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1348648529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: from='client.56714 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:00.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/880324815' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:42:00.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:00 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:42:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[29369]: pgmap v164: 332 pgs: 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:42:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[29369]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[29369]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:42:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[29369]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:42:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3642688511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-80182-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[40885]: pgmap v164: 332 pgs: 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:42:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[40885]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[40885]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:42:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[40885]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:42:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3642688511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-80182-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:01 smithi175 ceph-mon[33399]: pgmap v164: 332 pgs: 32 unknown, 300 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:42:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:01 smithi175 ceph-mon[33399]: from='client.56714 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi131-80207-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:01 smithi175 ceph-mon[33399]: from='client.34727 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi131-80450-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:42:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:01 smithi175 ceph-mon[33399]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:42:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3642688511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-80182-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:01.983 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-10T18:42:01.983 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-10T18:42:01.983 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-10T18:42:01.983 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: seed 80450 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (0 ms) 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (278 ms) 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (13 ms) 2024-09-10T18:42:01.984 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (9 ms) 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (8 ms) 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (5 ms) 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-10T18:42:01.985 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (11 ms) 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-10T18:42:01.986 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1460 ms) 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (368 ms) 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: done waiting, doing copies 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: done waiting 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60686 ms) 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-10T18:42:01.987 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4232 ms) 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-10T18:42:01.988 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-10T18:42:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3642688511' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-80182-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:42:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3642688511' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-80182-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:02.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:02.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3642688511' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi131-80182-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:42:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:02.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:02.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:02.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:03.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:42:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: pgmap v167: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:42:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/982731532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:03.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:03 smithi175 ceph-mon[33399]: from='client.47905 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: pgmap v167: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:42:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:03.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:42:03.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:03.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/982731532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:03.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:03.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[29369]: from='client.47905 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: pgmap v167: 300 pgs: 32 unknown, 268 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:42:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80450-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:42:03.807 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:03.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/982731532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:03.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:03.808 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-mon[40885]: from='client.47905 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:42:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:03] "GET /metrics HTTP/1.1" 200 235540 "" "Prometheus/2.43.0" 2024-09-10T18:42:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[29369]: from='client.47905 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[29369]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:42:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2309295392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-80182-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[40885]: from='client.47905 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[40885]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:42:04.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2309295392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-80182-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:04 smithi175 ceph-mon[33399]: from='client.47905 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi131-80207-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:04 smithi175 ceph-mon[33399]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:42:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2309295392' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-80182-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:42:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[29369]: pgmap v170: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2309295392' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-80182-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[29369]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[40885]: pgmap v170: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:05.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:05.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:05.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2309295392' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-80182-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:05.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:05 smithi131 ceph-mon[40885]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:42:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:05 smithi175 ceph-mon[33399]: pgmap v170: 332 pgs: 32 unknown, 32 creating+peering, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:05 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80450-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2309295392' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi131-80182-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:05 smithi175 ceph-mon[33399]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:42:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:42:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1791915646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:06.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:42:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1791915646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:06.832 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:06 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:42:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1791915646' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.56738 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:06.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:06.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:06 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: pgmap v173: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:42:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:42:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: pgmap v173: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:07.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: pgmap v173: 340 pgs: 72 unknown, 268 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:42:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.56738 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi131-80207-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi131-80182-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:42:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:07.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:07.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:07.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:07.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP ( api_snapshots_pp: Running main() from gmock_main.cc 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: seed 80551 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (2419 ms) 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-10T18:42:08.922 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2521 ms) 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2529 ms) 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2554 ms) 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4322 ms) 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (14345 ms total) 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (3944 ms) 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-10T18:42:08.923 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3965 ms) 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6036 ms) 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3932 ms) 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2256 ms) 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi131-80551-7 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17744 ms) 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (37877 ms total) 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-09-10T18:42:08.924 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-10T18:42:08.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-10T18:42:08.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-09-10T18:42:08.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-10T18:42:08.925 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5038 ms) 2024-09-10T18:42:08.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5046 ms total) 2024-09-10T18:42:08.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-09-10T18:42:08.928 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3005 ms) 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2010 ms) 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2254 ms) 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1767 ms) 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9036 ms total) 2024-09-10T18:42:08.929 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-09-10T18:42:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: pgmap v176: 300 pgs: 300 active+clean; 8.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:09.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:08 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:42:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:42:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:42:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: pgmap v176: 300 pgs: 300 active+clean; 8.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80450-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1333924954' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]: dispatch 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: pgmap v176: 300 pgs: 300 active+clean; 8.6 MiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:09.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:08 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:42:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:42:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2771137095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80207-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:10.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:42:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:42:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:42:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2771137095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80207-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi131-80182-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.56696 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi131-80551-15"}]': finished 2024-09-10T18:42:10.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2771137095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80207-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:10.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2771137095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80207-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:42:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:11.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:11.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:11.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:11.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:42:11.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:42:11.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:10 smithi175 ceph-mon[33399]: pgmap v179: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2771137095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80207-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[29369]: pgmap v179: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2771137095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi131-80207-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:11.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:42:11.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:42:11.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:10 smithi131 ceph-mon[40885]: pgmap v179: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:42:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:42:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:12.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:12.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:11 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:42:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:42:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3829015374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:12.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:11 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]: dispatch 2024-09-10T18:42:12.945 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] Global test environment set-up. 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.TooBig (6845 ms) 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (3918 ms) 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-10T18:42:12.946 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (2811 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2282 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3016 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3031 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (2963 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3017 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3020 ms) 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-10T18:42:12.947 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3031 ms) 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2956 ms) 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2976 ms) 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3011 ms) 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2991 ms) 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.Flush (3012 ms) 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-10T18:42:12.948 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2993 ms) 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2997 ms) 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3026 ms) 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3011 ms) 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3005 ms) 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3007 ms) 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-10T18:42:12.949 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3014 ms) 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3007 ms) 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3026 ms) 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3019 ms) 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 25 tests from LibRadosAio (78985 ms total) 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-10T18:42:12.950 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (6926 ms) 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2663739171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-80207-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:13.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[29369]: pgmap v182: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:42:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2663739171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-80207-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:12 smithi131 ceph-mon[40885]: pgmap v182: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.47902 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80450-24"}]': finished 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2663739171' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-80207-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1524226617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]: dispatch 2024-09-10T18:42:13.360 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:13.361 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:12 smithi175 ceph-mon[33399]: pgmap v182: 300 pgs: 40 unknown, 260 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:13.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:13] "GET /metrics HTTP/1.1" 200 235540 "" "Prometheus/2.43.0" 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2663739171' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-80207-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/617269513' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-80450-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-80182-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2663739171' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-80207-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/617269513' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-80450-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-80182-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:14.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:13 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2663739171' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi131-80207-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.56744 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi131-80182-26"}]': finished 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/617269513' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-80450-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-80182-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:14.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:14.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:13 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/617269513' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-80450-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-80182-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-80182-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: pgmap v185: 300 pgs: 32 creating+peering, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:42:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/617269513' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-80450-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-80182-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-80182-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: pgmap v185: 300 pgs: 32 creating+peering, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:42:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:14 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:42:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/617269513' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi131-80450-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi131-80182-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-80182-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: pgmap v185: 300 pgs: 32 creating+peering, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:42:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:14 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:42:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:42:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[29369]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:42:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1281156911' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-80207-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:42:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[40885]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:42:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1281156911' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-80207-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:15 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:15 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:42:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:15 smithi175 ceph-mon[33399]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:42:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1281156911' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-80207-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-80182-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1281156911' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-80207-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2879741030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[29369]: pgmap v188: 340 pgs: 72 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:42:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-80182-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1281156911' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-80207-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:42:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2879741030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:16 smithi131 ceph-mon[40885]: pgmap v188: 340 pgs: 72 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi131-80182-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1281156911' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi131-80207-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2879741030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: from='client.56786 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:16 smithi175 ceph-mon[33399]: pgmap v188: 340 pgs: 72 unknown, 268 active+clean; 600 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:42:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[29369]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[29369]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:42:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[40885]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[40885]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:42:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:17 smithi175 ceph-mon[33399]: from='client.56786 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi131-80450-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:17 smithi175 ceph-mon[33399]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:42:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:17 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:42:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:42:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075606317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.47959 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[29369]: pgmap v191: 299 pgs: 32 creating+peering, 267 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075606317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:42:19.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.47959 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:19.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:19.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:18 smithi131 ceph-mon[40885]: pgmap v191: 299 pgs: 32 creating+peering, 267 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075606317' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]: dispatch 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.47959 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:19.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:19.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:18 smithi175 ceph-mon[33399]: pgmap v191: 299 pgs: 32 creating+peering, 267 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 10 KiB/s wr, 24 op/s 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]': finished 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.47959 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3454656744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]': finished 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.47959 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3454656744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:20.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-13"}]': finished 2024-09-10T18:42:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.47959 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi131-80207-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:42:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3454656744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]: dispatch 2024-09-10T18:42:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.56795 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:20.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:42:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: pgmap v194: 267 pgs: 32 unknown, 235 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3739796189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-80207-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: pgmap v194: 267 pgs: 32 unknown, 235 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-10T18:42:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3739796189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-80207-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:21.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/922101039' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi131-80182-27"}]': finished 2024-09-10T18:42:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56795 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi131-80450-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: pgmap v194: 267 pgs: 32 unknown, 235 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi131-80182-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3739796189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-80207-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:21.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3739796189' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-80207-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3739796189' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-80207-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:42:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:22.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:22.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3739796189' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi131-80207-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:22.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-mon[33399]: pgmap v197: 299 pgs: 64 unknown, 235 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-mon[33399]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:42:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:23.678 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:42:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:42:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[29369]: pgmap v197: 299 pgs: 64 unknown, 235 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[29369]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:42:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[40885]: pgmap v197: 299 pgs: 64 unknown, 235 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi131-80182-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[40885]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:23.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:42:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:23] "GET /metrics HTTP/1.1" 200 237998 "" "Prometheus/2.43.0" 2024-09-10T18:42:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:42:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4121999583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-80207-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:42:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4121999583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-80207-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi131-80450-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4121999583' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-80207-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:25.408 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: pgmap v200: 315 pgs: 40 unknown, 8 creating+peering, 267 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-10T18:42:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]': finished 2024-09-10T18:42:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4121999583' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-80207-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:42:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]: dispatch 2024-09-10T18:42:25.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]: dispatch 2024-09-10T18:42:25.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:25.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:25.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:25.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:25.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: pgmap v200: 315 pgs: 40 unknown, 8 creating+peering, 267 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-10T18:42:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]': finished 2024-09-10T18:42:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4121999583' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-80207-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:42:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: pgmap v200: 315 pgs: 40 unknown, 8 creating+peering, 267 active+clean; 609 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-15"}]': finished 2024-09-10T18:42:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4121999583' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi131-80207-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]: dispatch 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]: dispatch 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:25.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]': finished 2024-09-10T18:42:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:26.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:26 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]': finished 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:26.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-15", "mode": "writeback"}]': finished 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:26.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4075862958' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3686562299' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:26.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:26 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:42:26.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComple: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-10T18:42:26.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-10T18:42:26.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4051 ms) 2024-09-10T18:42:26.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-10T18:42:26.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3999 ms) 2024-09-10T18:42:26.930 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-10T18:42:26.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3925 ms) 2024-09-10T18:42:26.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11975 ms total) 2024-09-10T18:42:26.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: 2024-09-10T18:42:26.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-10T18:42:26.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (99781 ms total) 2024-09-10T18:42:26.931 INFO:tasks.workunit.client.0.smithi131.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-10T18:42:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: pgmap v203: 259 pgs: 259 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:27.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:27.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:27.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:27.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:27.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:42:27.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:42:27.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:27.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:27.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2589206899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-80207-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:27.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:27.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:27.680 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:27.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:27.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:27.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:27.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:42:27.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:27 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:42:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: pgmap v203: 259 pgs: 259 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2589206899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-80207-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: pgmap v203: 259 pgs: 259 active+clean; 585 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:27.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56807 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi131-80182-28"}]': finished 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56756 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi131-80551-20"}]': finished 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1115613581' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]: dispatch 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2589206899' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-80207-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:42:27.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:27 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:42:27.934 INFO:tasks.workunit.client.0.smithi131.stdout:67103 ms total) 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (100 ms) 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (100 ms total) 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (42 ms) 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (6 ms) 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (48 ms total) 2024-09-10T18:42:27.935 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (31 ms) 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (19 ms) 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (50 ms total) 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (6 ms) 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (2 ms) 2024-09-10T18:42:27.936 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (8 ms total) 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1038 ms) 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1038 ms total) 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (100805 ms total) 2024-09-10T18:42:27.937 INFO:tasks.workunit.client.0.smithi131.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2589206899' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-80207-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:42:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: pgmap v206: 291 pgs: 291 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2589206899' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-80207-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: pgmap v206: 291 pgs: 291 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-09-10T18:42:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:28 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:29.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.56813 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi131-80450-36"}]': finished 2024-09-10T18:42:29.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2589206899' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi131-80207-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:29.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi131-80182-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:29.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.24715 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi131-80278-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/637384201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]: dispatch 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: pgmap v206: 291 pgs: 291 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "30"}]: dispatch 2024-09-10T18:42:29.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:28 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:42:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]': finished 2024-09-10T18:42:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-09-10T18:42:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:42:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]': finished 2024-09-10T18:42:30.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:30.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:29 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:42:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-15"}]': finished 2024-09-10T18:42:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi131-80278-1", "var": "pg_num_actual", "val": "30"}]': finished 2024-09-10T18:42:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:42:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:29 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3074266639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[29369]: pgmap v209: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3074266639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:30 smithi131 ceph-mon[40885]: pgmap v209: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi131-80182-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi131-80278-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:42:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3074266639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: from='client.56837 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:30 smithi175 ceph-mon[33399]: pgmap v209: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:31.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-10T18:42:31.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-10T18:42:31.911 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (6848 ms) 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (20023 ms) 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6026 ms) 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2997 ms) 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3083 ms) 2024-09-10T18:42:31.912 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2891 ms) 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2997 ms) 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3012 ms) 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2986 ms) 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3014 ms) 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3001 ms) 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-10T18:42:31.913 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (3019 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3009 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3008 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3250 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (2776 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3011 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3013 ms) 2024-09-10T18:42:31.914 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3155 ms) 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3009 ms) 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2791 ms) 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3020 ms) 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3007 ms) 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3168 ms) 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-10T18:42:31.915 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2778 ms) 2024-09-10T18:42:31.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-10T18:42:31.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3263 ms) 2024-09-10T18:42:31.916 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:42:32.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:32.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:42:32.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:32.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:32.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:32.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.56837 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi131-80207-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:42:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi131-80278-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:32.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:32.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:42:32.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:32.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:32.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:32.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAiory=11 expected=11 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:863748b0:::15:head 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:863748b0:::15:head expected=15:863748b0:::15:head 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:863748b0:::15:head -> 15 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=15 expected=15 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:6cac518f:::0:head 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:6cac518f:::0:head expected=15:6cac518f:::0:head 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:6cac518f:::0:head -> 0 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=0 expected=0 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:62a1935d:::14:head 2024-09-10T18:42:32.964 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:62a1935d:::14:head expected=15:62a1935d:::14:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:62a1935d:::14:head -> 14 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=14 expected=14 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:f905c69b:::2:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:f905c69b:::2:head expected=15:f905c69b:::2:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:f905c69b:::2:head -> 2 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=2 expected=2 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:e960b815:::9:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:e960b815:::9:head expected=15:e960b815:::9:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:e960b815:::9:head -> 9 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=9 expected=9 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:02547ec2:::1:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:02547ec2:::1:head expected=15:02547ec2:::1:head 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:02547ec2:::1:head -> 1 2024-09-10T18:42:32.965 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=1 expected=1 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:5c6b0b28:::7:head 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:5c6b0b28:::7:head expected=15:5c6b0b28:::7:head 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:5c6b0b28:::7:head -> 7 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=7 expected=7 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : seek to 15:566253c9:::13:head 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : cursor()=15:566253c9:::13:head expected=15:566253c9:::13:head 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: > 15:566253c9:::13:head -> 13 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: : entry=13 expected=13 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (201 ms) 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (75392 ms) 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-10T18:42:32.966 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 0/5 -> MIN 15:33333333::::head 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 1/5 -> 15:33333333::::head 15:66666666::::head 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 2/5 -> 15:66666666::::head 15:99999999::::head 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 3/5 -> 15:99999999::::head 15:cccccccc::::head 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: split 4/5 -> 15:cccccccc::::head MAX 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (18015 ms) 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 7 tests from LibRadosList (94109 ms total) 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1025 ms) 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo2,foo3 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-09-10T18:42:32.967 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo2 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo3 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo1,foo4,foo5 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo4 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo5 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo1 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset foo6,foo7 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo7 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: foo6 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo4 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo5 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo7 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns2:foo6 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: ns1:foo1 2024-09-10T18:42:32.968 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo1 2024-09-10T18:42:32.969 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo2 2024-09-10T18:42:32.969 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: :foo3 2024-09-10T18:42:32.969 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (30 ms) 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: pgmap v212: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2788118315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-80207-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: pgmap v212: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:42:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2788118315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-80207-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:33.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:42:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: pgmap v212: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:42:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2664969764' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]: dispatch 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2788118315' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-80207-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:42:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:33] "GET /metrics HTTP/1.1" 200 230735 "" "Prometheus/2.43.0" 2024-09-10T18:42:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2788118315' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-80207-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]': finished 2024-09-10T18:42:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:42:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-80182-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2788118315' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-80207-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]': finished 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]: dispatch 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]: dispatch 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-80182-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.56831 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi131-80182-29"}]': finished 2024-09-10T18:42:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2788118315' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi131-80207-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-17"}]': finished 2024-09-10T18:42:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:42:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]: dispatch 2024-09-10T18:42:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]: dispatch 2024-09-10T18:42:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-80182-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: pgmap v215: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]': finished 2024-09-10T18:42:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-80182-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-80182-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: pgmap v215: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]': finished 2024-09-10T18:42:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-80182-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:42:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:35.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-80182-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:35.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:35.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:35.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:35.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: pgmap v215: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:35.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:35.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-17", "mode": "writeback"}]': finished 2024-09-10T18:42:35.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:35.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi131-80182-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:35.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:42:35.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/126073007' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:35.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-80182-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:35.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]: dispatch 2024-09-10T18:42:35.581 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:35.582 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:42:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:37.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3786034198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-80207-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: pgmap v218: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:37.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:36 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:37.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:37.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:37.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:42:37.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:37.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:37.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3786034198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-80207-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: pgmap v218: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: from='client.56843 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi131-80278-2"}]': finished 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:37.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]: dispatch 2024-09-10T18:42:37.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3786034198' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-80207-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:37.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: pgmap v218: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:37.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:36 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-80182-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]': finished 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3786034198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-80207-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3483443094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:37 smithi175 ceph-mon[33399]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-80182-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]': finished 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3786034198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-80207-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3483443094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:38.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[40885]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi131-80182-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-17"}]': finished 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3786034198' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi131-80207-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3483443094' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:38.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:37 smithi131 ceph-mon[29369]: from='client.56867 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:38.959 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 1 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 10 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 13 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 7 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 14 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 0 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 15 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 11 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 5 0 2024-09-10T18:42:38.960 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 8 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 6 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 3 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 4 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 12 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 9 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2 0 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (38 ms) 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 3 tests from LibRadosListEC (1093 ms total) 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-10T18:42:38.961 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-10T18:42:38.962 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2788 ms) 2024-09-10T18:42:38.962 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] 1 test from LibRadosListNP (2788 ms total) 2024-09-10T18:42:38.962 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: 2024-09-10T18:42:38.962 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [----------] Global test environment tear-down 2024-09-10T18:42:38.962 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [==========] 11 tests from 3 test suites ran. (111886 ms total) 2024-09-10T18:42:38.962 INFO:tasks.workunit.client.0.smithi131.stdout: api_list: [ PASSED ] 11 tests. 2024-09-10T18:42:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[29369]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[29369]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:42:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3483443094' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:42:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[29369]: from='client.56867 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:42:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[29369]: pgmap v221: 236 pgs: 236 active+clean; 595 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-09-10T18:42:39.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[40885]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:39.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[40885]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:42:39.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3483443094' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:42:39.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[40885]: from='client.56867 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:42:39.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:38 smithi131 ceph-mon[40885]: pgmap v221: 236 pgs: 236 active+clean; 595 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-09-10T18:42:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:38 smithi175 ceph-mon[33399]: from='client.56867 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80278-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:38 smithi175 ceph-mon[33399]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:42:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3483443094' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:42:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:38 smithi175 ceph-mon[33399]: from='client.56867 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:42:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:38 smithi175 ceph-mon[33399]: pgmap v221: 236 pgs: 236 active+clean; 595 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.2 KiB/s rd, 3.5 KiB/s wr, 2 op/s 2024-09-10T18:42:39.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: from='client.56867 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:42:39.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:42:39.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/883756371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:39.963 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: from='client.56867 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:42:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:42:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/883756371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: from='client.48040 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: from='client.56867 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi131-80278-3","pool2":"test-rados-api-smithi131-80278-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/883756371' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: from='client.48040 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: from='client.48040 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.48040 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: pgmap v224: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:41.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.48040 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: pgmap v224: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:42:41.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:41.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.48040 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi131-80207-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:42:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]: dispatch 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: pgmap v224: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2714197337' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi131-80182-30"}]': finished 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:41.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]': finished 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2950023291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.56879 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]': finished 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:42:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2950023291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:42.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]: dispatch 2024-09-10T18:42:42.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.56879 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:42.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-19"}]': finished 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi131-80182-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2950023291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]: dispatch 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.56879 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]: dispatch 2024-09-10T18:42:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:43 smithi131 ceph-mon[29369]: pgmap v227: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:43.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:43 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:43.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:43 smithi131 ceph-mon[40885]: pgmap v227: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:43.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:43 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:43.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:43 smithi175 ceph-mon[33399]: pgmap v227: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:43.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:43 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:43.680 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:42:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:42:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:42:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:43] "GET /metrics HTTP/1.1" 200 230735 "" "Prometheus/2.43.0" 2024-09-10T18:42:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.56879 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]': finished 2024-09-10T18:42:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:42:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2950023291' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:42:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.56879 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.56879 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.56879 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:44.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]': finished 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2950023291' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.56879 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.56879 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:44.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:44 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.56879 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi131-80207-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-19", "mode": "writeback"}]': finished 2024-09-10T18:42:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2950023291' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.56879 ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi131-80182-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.56879 ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi131-80207-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:44.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:44 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]: dispatch 2024-09-10T18:42:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:42:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:45 smithi175 ceph-mon[33399]: pgmap v230: 268 pgs: 8 unknown, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-09-10T18:42:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:45 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:45 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]': finished 2024-09-10T18:42:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:45 smithi175 ceph-mon[33399]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[29369]: pgmap v230: 268 pgs: 8 unknown, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]': finished 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[29369]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[40885]: pgmap v230: 268 pgs: 8 unknown, 260 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.5 KiB/s wr, 7 op/s 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:45.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-19"}]': finished 2024-09-10T18:42:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:45 smithi131 ceph-mon[40885]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:42:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[29369]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:42:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3641177608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[29369]: from='client.56885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[29369]: pgmap v233: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[40885]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:42:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3641177608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[40885]: from='client.56885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:46 smithi131 ceph-mon[40885]: pgmap v233: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:46 smithi175 ceph-mon[33399]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:42:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3641177608' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:46 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:46 smithi175 ceph-mon[33399]: from='client.56885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:46 smithi175 ceph-mon[33399]: pgmap v233: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='client.56885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:42:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:48.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='client.56885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:48.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='client.56885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi131-80207-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/278388012' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]: dispatch 2024-09-10T18:42:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:42:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-80182-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[29369]: pgmap v236: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-80182-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:48 smithi131 ceph-mon[40885]: pgmap v236: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.56876 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi131-80182-31"}]': finished 2024-09-10T18:42:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:42:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-80182-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:48 smithi175 ceph-mon[33399]: pgmap v236: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-80182-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-80182-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:50.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/365351036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[29369]: from='client.48067 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-80182-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-80182-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/365351036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:50.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:50 smithi131 ceph-mon[40885]: from='client.48067 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi131-80182-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:42:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-80182-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:50.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:50.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/365351036' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:50.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:50 smithi175 ceph-mon[33399]: from='client.48067 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]': finished 2024-09-10T18:42:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.48067 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:42:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]: dispatch 2024-09-10T18:42:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]: dispatch 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-80182-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]': finished 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]': finished 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.48067 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]: dispatch 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]: dispatch 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-80182-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]': finished 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:42:51.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-21"}]': finished 2024-09-10T18:42:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.48067 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi131-80207-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:42:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]: dispatch 2024-09-10T18:42:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]: dispatch 2024-09-10T18:42:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: pgmap v239: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:42:51.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:42:51.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi131-80182-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:51.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-21", "mode": "writeback"}]': finished 2024-09-10T18:42:51.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:42:51.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/466784401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[29369]: from='client.48073 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/466784401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:52 smithi131 ceph-mon[40885]: from='client.48073 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:42:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:42:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:42:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/466784401' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]: dispatch 2024-09-10T18:42:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:52 smithi175 ceph-mon[33399]: from='client.48073 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[29369]: pgmap v242: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]': finished 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[29369]: from='client.48073 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[29369]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:53.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[40885]: pgmap v242: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]': finished 2024-09-10T18:42:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[40885]: from='client.48073 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[40885]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:42:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:53.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:42:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-mon[33399]: pgmap v242: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:42:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:42:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-21"}]': finished 2024-09-10T18:42:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-mon[33399]: from='client.48073 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:53.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-mon[33399]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:42:53.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:42:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:53] "GET /metrics HTTP/1.1" 200 230718 "" "Prometheus/2.43.0" 2024-09-10T18:42:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[29369]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:42:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3770768634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[40885]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:42:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3770768634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:54 smithi175 ceph-mon[33399]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:42:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]: dispatch 2024-09-10T18:42:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3770768634' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:42:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: pgmap v245: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:42:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3770768634' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-80182-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: pgmap v245: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3770768634' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-80182-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: pgmap v245: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3746382355' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi131-80182-32"}]': finished 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3770768634' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:55.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:55.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-80182-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:42:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-80182-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[29369]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:42:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-80182-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3271849555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[29369]: pgmap v248: 324 pgs: 96 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:42:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-80182-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[40885]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:42:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-80182-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3271849555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:57 smithi131 ceph-mon[40885]: pgmap v248: 324 pgs: 96 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:42:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi131-80182-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:42:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:57 smithi175 ceph-mon[33399]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:42:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-80182-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:42:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3271849555' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:42:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:57 smithi175 ceph-mon[33399]: pgmap v248: 324 pgs: 96 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:42:58.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3271849555' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[29369]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3271849555' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[40885]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:58.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3271849555' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi131-80207-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:42:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:58 smithi175 ceph-mon[33399]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:42:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-80182-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: pgmap v251: 300 pgs: 8 creating+peering, 292 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:42:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]': finished 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]: dispatch 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]: dispatch 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-80182-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: pgmap v251: 300 pgs: 8 creating+peering, 292 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:42:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]': finished 2024-09-10T18:42:59.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:42:59.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]: dispatch 2024-09-10T18:42:59.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:42:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]: dispatch 2024-09-10T18:42:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi131-80182-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:42:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:42:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:42:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:42:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:42:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: pgmap v251: 300 pgs: 8 creating+peering, 292 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:42:59.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-23"}]': finished 2024-09-10T18:42:59.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:42:59.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]: dispatch 2024-09-10T18:42:59.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:42:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]: dispatch 2024-09-10T18:43:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]': finished 2024-09-10T18:43:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[29369]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:43:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:43:00.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]': finished 2024-09-10T18:43:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[40885]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:43:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:43:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:00 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-23", "mode": "writeback"}]': finished 2024-09-10T18:43:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:00 smithi175 ceph-mon[33399]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:43:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[29369]: pgmap v254: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[29369]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2289940620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[29369]: from='client.56924 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:01.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[40885]: pgmap v254: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:43:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:43:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[40885]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:43:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2289940620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:43:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:01 smithi131 ceph-mon[40885]: from='client.56924 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:01 smithi175 ceph-mon[33399]: pgmap v254: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:43:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:43:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:01 smithi175 ceph-mon[33399]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:43:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2289940620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]: dispatch 2024-09-10T18:43:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:01 smithi175 ceph-mon[33399]: from='client.56924 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.56924 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.56924 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:43:02.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:43:02.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:02 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:43:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1004673093' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi131-80182-33"}]': finished 2024-09-10T18:43:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.56924 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:02.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:02.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:43:02.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:43:02.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:02 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]: dispatch 2024-09-10T18:43:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-80182-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: pgmap v257: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]': finished 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-80182-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2188298704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi131-80182-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[29369]: from='client.48103 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-80182-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: pgmap v257: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]': finished 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-80182-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:43:03.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2188298704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:03.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi131-80182-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-mon[40885]: from='client.48103 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-80182-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: pgmap v257: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-23"}]': finished 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi131-80182-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:03.407 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:43:03.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2188298704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:03.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi131-80182-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:03.408 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-mon[33399]: from='client.48103 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:03.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:43:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:43:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:43:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:03] "GET /metrics HTTP/1.1" 200 230760 "" "Prometheus/2.43.0" 2024-09-10T18:43:04.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:04.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: pgmap v260: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:43:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: from='client.48103 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:43:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi131-80182-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:43:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2177069684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:05 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: pgmap v260: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:43:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: from='client.48103 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:43:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi131-80182-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:43:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2177069684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: pgmap v260: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: from='client.48103 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi131-80182-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2177069684' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: from='client.56939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:05 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: pgmap v263: 332 pgs: 72 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/589350387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: pgmap v263: 332 pgs: 72 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/589350387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: from='client.56939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:43:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: pgmap v263: 332 pgs: 72 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:43:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:43:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/589350387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/589350387' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/589350387' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:08.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:08.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/589350387' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:43:08.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:08.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:08.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]: dispatch 2024-09-10T18:43:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: pgmap v266: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]': finished 2024-09-10T18:43:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:43:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]: dispatch 2024-09-10T18:43:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1902477321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: pgmap v266: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]': finished 2024-09-10T18:43:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1902477321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:09.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:09 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: pgmap v266: 356 pgs: 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-25"}]': finished 2024-09-10T18:43:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3761263878' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi131-80182-34"}]': finished 2024-09-10T18:43:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:43:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1902477321' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.56951 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:09.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:09 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]': finished 2024-09-10T18:43:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]': finished 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:10.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:10.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-25", "mode": "writeback"}]': finished 2024-09-10T18:43:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.56951 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi131-80207-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi131-80182-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:43:10.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:10.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:10.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:10.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[29369]: pgmap v269: 388 pgs: 32 unknown, 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[29369]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:43:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[40885]: pgmap v269: 388 pgs: 32 unknown, 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[40885]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:43:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:11 smithi175 ceph-mon[33399]: pgmap v269: 388 pgs: 32 unknown, 356 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:11 smithi175 ceph-mon[33399]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:43:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]: dispatch 2024-09-10T18:43:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]': finished 2024-09-10T18:43:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[29369]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:43:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]': finished 2024-09-10T18:43:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:12 smithi131 ceph-mon[40885]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:43:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:12 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:12 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi131-80182-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-25"}]': finished 2024-09-10T18:43:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:12 smithi175 ceph-mon[33399]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:13 smithi131 ceph-mon[29369]: pgmap v272: 332 pgs: 8 unknown, 324 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:13 smithi131 ceph-mon[29369]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:13 smithi131 ceph-mon[40885]: pgmap v272: 332 pgs: 8 unknown, 324 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:13 smithi131 ceph-mon[40885]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:43:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:13 smithi175 ceph-mon[33399]: pgmap v272: 332 pgs: 8 unknown, 324 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:13 smithi175 ceph-mon[33399]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:43:13.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:43:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:43:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:43:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:13] "GET /metrics HTTP/1.1" 200 230760 "" "Prometheus/2.43.0" 2024-09-10T18:43:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:43:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-mon[33399]: pgmap v275: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-mon[33399]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:43:15.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:15.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:15 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[29369]: pgmap v275: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[29369]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[40885]: pgmap v275: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[40885]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:43:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2164083468' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:15 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1666204176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-80207-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[29369]: pgmap v278: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1666204176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-80207-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:17.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:17.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:16 smithi131 ceph-mon[40885]: pgmap v278: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.56957 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi131-80182-35"}]': finished 2024-09-10T18:43:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:43:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1666204176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-80207-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:17.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:16 smithi175 ceph-mon[33399]: pgmap v278: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1666204176' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-80207-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1666204176' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-80207-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:18.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1666204176' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi131-80207-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:43:18.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:18.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:18.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:18.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:18.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:18.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]': finished 2024-09-10T18:43:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]: dispatch 2024-09-10T18:43:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:43:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]: dispatch 2024-09-10T18:43:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: pgmap v281: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:43:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]': finished 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3629379966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[29369]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]': finished 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]: dispatch 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:43:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]: dispatch 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: pgmap v281: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]': finished 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3629379966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:19 smithi131 ceph-mon[40885]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-27"}]': finished 2024-09-10T18:43:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]: dispatch 2024-09-10T18:43:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:43:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]: dispatch 2024-09-10T18:43:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: pgmap v281: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-09-10T18:43:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi131-80182-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-27", "mode": "writeback"}]': finished 2024-09-10T18:43:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:43:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3629379966' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:19.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:19 smithi175 ceph-mon[33399]: from='client.56975 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[29369]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[29369]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:43:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:20.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[40885]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[40885]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:43:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:20 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:20 smithi175 ceph-mon[33399]: from='client.56975 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi131-80207-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:20 smithi175 ceph-mon[33399]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:43:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: pgmap v284: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: pgmap v284: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:21.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: pgmap v284: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]: dispatch 2024-09-10T18:43:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:21.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]': finished 2024-09-10T18:43:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]': finished 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:22.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:43:22.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:22.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:22.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:22 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-27"}]': finished 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2383191635' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:22 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]: dispatch 2024-09-10T18:43:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-mon[29369]: pgmap v287: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-mon[29369]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-mon[29369]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:43:23.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-mon[40885]: pgmap v287: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:23.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-mon[40885]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:23.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-mon[40885]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:43:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:23 smithi175 ceph-mon[33399]: pgmap v287: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:23 smithi175 ceph-mon[33399]: from='client.56969 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi131-80182-36"}]': finished 2024-09-10T18:43:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:23 smithi175 ceph-mon[33399]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:43:23.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:43:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:43:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:43:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:23] "GET /metrics HTTP/1.1" 200 230776 "" "Prometheus/2.43.0" 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:24.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:24.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:24 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi131-80182-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:24.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:24 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:43:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[29369]: pgmap v290: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[29369]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:43:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[40885]: pgmap v290: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:25 smithi131 ceph-mon[40885]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:43:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:25 smithi175 ceph-mon[33399]: pgmap v290: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:25 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:25 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:25 smithi175 ceph-mon[33399]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:43:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:43:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: pgmap v293: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: pgmap v293: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:27.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:27.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:43:27.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:27.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:27.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:26 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi131-80182-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:43:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: pgmap v293: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:43:27.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]: dispatch 2024-09-10T18:43:27.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:27.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:26 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:29.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:29.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:29.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:29.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:29.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:28 smithi175 ceph-mon[33399]: pgmap v296: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-10T18:43:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:43:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:29.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[29369]: pgmap v296: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2125031517' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-46"}]': finished 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:29.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:29.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:28 smithi131 ceph-mon[40885]: pgmap v296: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]': finished 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]: dispatch 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]: dispatch 2024-09-10T18:43:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:30.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:30 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]': finished 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]: dispatch 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]: dispatch 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-29"}]': finished 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi131-80207-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]: dispatch 2024-09-10T18:43:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1302736006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:30.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]: dispatch 2024-09-10T18:43:30.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:43:30.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:43:30.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:30 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:43:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]': finished 2024-09-10T18:43:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:31.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: pgmap v299: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:43:31.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:31.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:31.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:43:31.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:31 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]': finished 2024-09-10T18:43:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:43:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: pgmap v299: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-29", "mode": "writeback"}]': finished 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.56987 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi131-80182-37"}]': finished 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: pgmap v299: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:43:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi131-80207-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:31.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi131-80182-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:31.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:43:31.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:31.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:31 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:32 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:32.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:32 smithi175 ceph-mon[33399]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:43:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:32 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:32 smithi131 ceph-mon[29369]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:43:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:32 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:32 smithi131 ceph-mon[40885]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:43:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: pgmap v302: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:43:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:33.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:33.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: pgmap v302: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:33.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:33.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:43:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: pgmap v302: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi131-80182-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:43:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:43:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:33] "GET /metrics HTTP/1.1" 200 230768 "" "Prometheus/2.43.0" 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:34.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]: dispatch 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]: dispatch 2024-09-10T18:43:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: pgmap v305: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]': finished 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-80207-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: pgmap v305: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]': finished 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-80207-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: pgmap v305: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/487457178' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi131-80207-47"}]': finished 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-29"}]': finished 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:35.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:35.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:35.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:35.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-80207-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:37.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:37.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-80207-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:37.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:43:37.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-80207-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:37.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:37.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:37.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: pgmap v308: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:43:37.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:37 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-80207-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-80207-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: pgmap v308: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:43:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi131-80207-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-80207-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3084104045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]: dispatch 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: pgmap v308: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-09-10T18:43:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:37 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:38.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:38.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:43:38.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-80207-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:38.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:43:38.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:38.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:38.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:38.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-80207-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:43:38.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.56999 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi131-80182-38"}]': finished 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:38.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi131-80207-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi131-80182-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:38.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:39 smithi175 ceph-mon[33399]: pgmap v311: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:39 smithi175 ceph-mon[33399]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:43:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[29369]: pgmap v311: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[29369]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[40885]: pgmap v311: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:39.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[40885]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:43:39.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:39.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]': finished 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]: dispatch 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]: dispatch 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[29369]: pgmap v314: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]': finished 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]: dispatch 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]: dispatch 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:40 smithi131 ceph-mon[40885]: pgmap v314: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi131-80182-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-31"}]': finished 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]: dispatch 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]: dispatch 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:40 smithi175 ceph-mon[33399]: pgmap v314: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]': finished 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]': finished 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-31", "mode": "writeback"}]': finished 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]: dispatch 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/808198467' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi131-80207-48"}]': finished 2024-09-10T18:43:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:43:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-80207-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: pgmap v317: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]': finished 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-80207-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-80207-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-80207-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: pgmap v317: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:43.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]': finished 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-80207-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-80207-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:42 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:43:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]: dispatch 2024-09-10T18:43:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-80207-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: pgmap v317: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-31"}]': finished 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi131-80207-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-80207-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2541497759' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:42 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]: dispatch 2024-09-10T18:43:43.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:43:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:43:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:43:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:43] "GET /metrics HTTP/1.1" 200 230768 "" "Prometheus/2.43.0" 2024-09-10T18:43:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:44 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:44 smithi131 ceph-mon[29369]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:44.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:44 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:44.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:44.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:44 smithi131 ceph-mon[40885]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:44 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:44 smithi175 ceph-mon[33399]: from='client.57011 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi131-80182-39"}]': finished 2024-09-10T18:43:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:43:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: pgmap v320: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-80207-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:45.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:45 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: pgmap v320: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-80207-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: pgmap v320: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi131-80207-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi131-80182-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:45.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:45 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[29369]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:43:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:47.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[29369]: pgmap v323: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[40885]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:43:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:47.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:46 smithi131 ceph-mon[40885]: pgmap v323: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:46 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:46 smithi175 ceph-mon[33399]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:43:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:46 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:46 smithi175 ceph-mon[33399]: pgmap v323: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:48.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:48.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi131-80182-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:43:48.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:43:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]': finished 2024-09-10T18:43:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:43:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]: dispatch 2024-09-10T18:43:49.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]: dispatch 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: pgmap v326: 236 pgs: 236 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]': finished 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]': finished 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:49.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]: dispatch 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]: dispatch 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: pgmap v326: 236 pgs: 236 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]': finished 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:49.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:48 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-33"}]': finished 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:43:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: pgmap v326: 236 pgs: 236 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3696488092' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi131-80207-49"}]': finished 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-33", "mode": "writeback"}]': finished 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:48 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:50.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:50.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:43:50.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:50.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:49 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:50.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:50.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi131-80207-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1900601689' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:50.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:49 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]: dispatch 2024-09-10T18:43:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:51 smithi131 ceph-mon[29369]: pgmap v329: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:51 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:51.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:51 smithi131 ceph-mon[40885]: pgmap v329: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:51.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:51 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:51 smithi175 ceph-mon[33399]: pgmap v329: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:43:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:51 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:43:52.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]': finished 2024-09-10T18:43:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:43:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:52.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:52.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:52.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:52.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:43:52.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:52.580 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]': finished 2024-09-10T18:43:52.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:43:52.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:52.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.581 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:52.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:52.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:52.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:52.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:43:52.582 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.583 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:52 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.57023 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi131-80182-40"}]': finished 2024-09-10T18:43:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-33"}]': finished 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:52.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:43:52.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi131-80207-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:52.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi131-80182-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:43:52.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:43:52.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:52.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:52 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[29369]: pgmap v332: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[29369]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:43:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[40885]: pgmap v332: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[40885]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:43:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:53.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:53.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:43:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:53 smithi175 ceph-mon[33399]: pgmap v332: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:53 smithi175 ceph-mon[33399]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:53 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:43:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:53] "GET /metrics HTTP/1.1" 200 230798 "" "Prometheus/2.43.0" 2024-09-10T18:43:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:43:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:54.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:54 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi131-80182-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:54 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:43:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:43:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:43:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: pgmap v335: 236 pgs: 32 creating+peering, 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:43:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:43:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:43:55.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:55 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: pgmap v335: 236 pgs: 32 creating+peering, 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:43:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: pgmap v335: 236 pgs: 32 creating+peering, 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1046348809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:43:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:55 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]: dispatch 2024-09-10T18:43:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:43:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:43:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]: dispatch 2024-09-10T18:43:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]: dispatch 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: pgmap v338: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]: dispatch 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]: dispatch 2024-09-10T18:43:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:57.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: pgmap v338: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:57.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:56 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: from='client.48190 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi131-80207-50"}]': finished 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]: dispatch 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]: dispatch 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: pgmap v338: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:43:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:56 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:43:57.976 INFO:tasks.workunit.client.0.smithi131.stdout:te 2024-09-10T18:43:57.977 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7289 ms) 2024-09-10T18:43:57.977 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-10T18:43:57.977 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6695 ms) 2024-09-10T18:43:57.977 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-10T18:43:57.977 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6994 ms) 2024-09-10T18:43:57.977 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7034 ms) 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7020 ms) 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7043 ms) 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7010 ms) 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6896 ms) 2024-09-10T18:43:57.978 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7013 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7259 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6973 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6996 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7015 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (6780 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7014 ms) 2024-09-10T18:43:57.979 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111957 ms total) 2024-09-10T18:43:57.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: 2024-09-10T18:43:57.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [----------] Global test environment tear-down 2024-09-10T18:43:57.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (190942 ms total) 2024-09-10T18:43:57.980 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio: [ PASSED ] 41 tests. 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]': finished 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/913658454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]': finished 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/913658454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-35", "mode": "writeback"}]': finished 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/267800565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]: dispatch 2024-09-10T18:43:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/913658454' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:43:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[29369]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/913658454' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[29369]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:43:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[29369]: pgmap v341: 260 pgs: 260 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:59.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[40885]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/913658454' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[40885]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:43:59.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:58 smithi131 ceph-mon[40885]: pgmap v341: 260 pgs: 260 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:43:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:58 smithi175 ceph-mon[33399]: from='client.57035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi131-80182-41"}]': finished 2024-09-10T18:43:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/913658454' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi131-80207-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:43:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:58 smithi175 ceph-mon[33399]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:43:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:58 smithi175 ceph-mon[33399]: pgmap v341: 260 pgs: 260 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:44:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[29369]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:44:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:00.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[40885]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:44:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:00.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:43:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:59 smithi175 ceph-mon[33399]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:44:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:00.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:43:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-80207-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:00 smithi131 ceph-mon[29369]: pgmap v344: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-80207-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:01.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:01 smithi131 ceph-mon[40885]: pgmap v344: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:01 smithi175 ceph-mon[33399]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-80207-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:01.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:01 smithi175 ceph-mon[33399]: pgmap v344: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:44:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:02 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:44:02.336 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:02 smithi131 ceph-mon[29369]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:44:02.336 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:02 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:02.336 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:02 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:44:02.336 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:02 smithi131 ceph-mon[40885]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:44:02.336 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:02 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:02 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:44:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:02 smithi175 ceph-mon[33399]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:44:02.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:02 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-80207-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[29369]: pgmap v347: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-80207-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:03.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-mon[40885]: pgmap v347: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi131-80207-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-mon[33399]: pgmap v347: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:03.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:44:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:44:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:44:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:03] "GET /metrics HTTP/1.1" 200 230839 "" "Prometheus/2.43.0" 2024-09-10T18:44:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[29369]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:44:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[40885]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:04 smithi175 ceph-mon[33399]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:44:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]: dispatch 2024-09-10T18:44:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:44:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:44:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:44:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:44:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:44:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-35"}]': finished 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:44:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]: dispatch 2024-09-10T18:44:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:44:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: pgmap v353: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: pgmap v353: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:07.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2780416731' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi131-80207-52"}]': finished 2024-09-10T18:44:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:07.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: pgmap v353: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:07.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:08 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:08 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:08.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:08 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]': finished 2024-09-10T18:44:09.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:44:09.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]: dispatch 2024-09-10T18:44:09.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]: dispatch 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]': finished 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:44:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]: dispatch 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]: dispatch 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: pgmap v356: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi131-80207-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-37"}]': finished 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]: dispatch 2024-09-10T18:44:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]: dispatch 2024-09-10T18:44:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:44:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]': finished 2024-09-10T18:44:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[29369]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:44:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:44:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]': finished 2024-09-10T18:44:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[40885]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:44:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:10.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:10 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:44:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-37", "mode": "writeback"}]': finished 2024-09-10T18:44:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:10 smithi175 ceph-mon[33399]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:44:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: pgmap v359: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: pgmap v359: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:11 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: pgmap v359: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]: dispatch 2024-09-10T18:44:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:11 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]': finished 2024-09-10T18:44:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[29369]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:44:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]': finished 2024-09-10T18:44:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[40885]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:44:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:12 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:12 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-37"}]': finished 2024-09-10T18:44:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:12 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:12 smithi175 ceph-mon[33399]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:44:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4023933719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:12 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]: dispatch 2024-09-10T18:44:13.478 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: pgmap v362: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:13.479 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: pgmap v362: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: pgmap v362: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.57053 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi131-80207-53"}]': finished 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:13.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:44:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:44:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:44:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:13] "GET /metrics HTTP/1.1" 200 230839 "" "Prometheus/2.43.0" 2024-09-10T18:44:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:44:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:14.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:14 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi131-80207-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:14.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:14 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:44:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:15 smithi175 ceph-mon[33399]: pgmap v365: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:15 smithi175 ceph-mon[33399]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:44:15.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[29369]: pgmap v365: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[29369]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:44:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[40885]: pgmap v365: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[40885]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:44:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[29369]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:44:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[29369]: pgmap v368: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[40885]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:44:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:16 smithi131 ceph-mon[40885]: pgmap v368: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:16 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi131-80207-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:16 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:16 smithi175 ceph-mon[33399]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:44:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:16 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:16 smithi175 ceph-mon[33399]: pgmap v368: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]': finished 2024-09-10T18:44:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[29369]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:44:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]: dispatch 2024-09-10T18:44:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]: dispatch 2024-09-10T18:44:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]': finished 2024-09-10T18:44:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[40885]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:44:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]: dispatch 2024-09-10T18:44:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]: dispatch 2024-09-10T18:44:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:17 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-39"}]': finished 2024-09-10T18:44:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:17 smithi175 ceph-mon[33399]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:44:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]: dispatch 2024-09-10T18:44:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:17 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]: dispatch 2024-09-10T18:44:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]': finished 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[29369]: pgmap v371: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]': finished 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:19.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:18 smithi131 ceph-mon[40885]: pgmap v371: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-10T18:44:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:44:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-39", "mode": "writeback"}]': finished 2024-09-10T18:44:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:44:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:19.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:19.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:18 smithi175 ceph-mon[33399]: pgmap v371: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:20.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3570934504' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]: dispatch 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]: dispatch 2024-09-10T18:44:20.989 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-10T18:44:20.989 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: seed 80444 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (295 ms) 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (11737 ms) 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (43 ms) 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (12076 ms total) 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-10T18:44:20.990 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7310 ms) 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7666 ms) 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9987 ms) 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [4] 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [5,4] 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [6,5,4] 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting some heads 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 7 2024-09-10T18:44:20.991 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: waiting for scrubs... 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: done waiting 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46080 ms) 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9966 ms) 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7993 ms) 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-10T18:44:20.992 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8022 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8043 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10270 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8657 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10266 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12751 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8232 ms) 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-10T18:44:20.993 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7783 ms) 2024-09-10T18:44:20.994 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-10T18:44:20.994 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13066 ms) 2024-09-10T18:44:20.994 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-10T18:44:20.994 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7916 ms) 2024-09-10T18:44:20.994 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-10T18:44:20.994 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8045 ms) 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]': finished 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-80207-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[29369]: pgmap v374: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]': finished 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-80207-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:21.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:20 smithi131 ceph-mon[40885]: pgmap v374: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-10T18:44:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:44:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: from='client.57059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi131-80207-54"}]': finished 2024-09-10T18:44:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-39"}]': finished 2024-09-10T18:44:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:44:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:21.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-80207-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:21.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:20 smithi175 ceph-mon[33399]: pgmap v374: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 108 op/s 2024-09-10T18:44:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-80207-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:22 smithi131 ceph-mon[29369]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:44:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-80207-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-80207-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:22 smithi131 ceph-mon[40885]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:44:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-80207-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi131-80207-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:22 smithi175 ceph-mon[33399]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:44:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-80207-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: pgmap v377: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-80207-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: pgmap v377: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-80207-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:23.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:44:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:44:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: pgmap v377: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi131-80207-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:44:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:23.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:24.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:44:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:23] "GET /metrics HTTP/1.1" 200 230780 "" "Prometheus/2.43.0" 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[29369]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[40885]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:44:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:44:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:24 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:44:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:24 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:24 smithi175 ceph-mon[33399]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:44:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:44:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:24 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:44:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-mon[33399]: pgmap v380: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:44:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-mon[33399]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:44:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[29369]: pgmap v380: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:44:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[29369]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:44:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[40885]: pgmap v380: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:44:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[40885]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:44:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: pgmap v383: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: pgmap v383: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:27.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:26 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]: dispatch 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: pgmap v383: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:26 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:28.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:28.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:44:28.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:44:28.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:28.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:28.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:28.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:44:28.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:44:28.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:28.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:28 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1653273219' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi131-80207-55"}]': finished 2024-09-10T18:44:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:44:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:44:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:28.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:28 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[29369]: pgmap v386: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]: dispatch 2024-09-10T18:44:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:29 smithi131 ceph-mon[40885]: pgmap v386: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi131-80207-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:44:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:29.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]: dispatch 2024-09-10T18:44:29.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]: dispatch 2024-09-10T18:44:29.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:29 smithi175 ceph-mon[33399]: pgmap v386: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]': finished 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[29369]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[29369]: osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]': finished 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[40885]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:30.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:30 smithi131 ceph-mon[40885]: osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:44:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-41"}]': finished 2024-09-10T18:44:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:30 smithi175 ceph-mon[33399]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:44:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:30 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi131-80207-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:30 smithi175 ceph-mon[33399]: osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:44:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: pgmap v389: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: pgmap v389: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:31.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: pgmap v389: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:44:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:44:31.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:44:31.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:44:31.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:31.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:33.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:44:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-80444-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:44:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:33.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:33.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:33.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:33.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[40885]: pgmap v392: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:33.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:33.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:44:33.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:33.330 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-80444-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:44:33.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:33.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:33.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:33.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:33.331 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-mon[29369]: pgmap v392: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-80444-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:33.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-mon[33399]: pgmap v392: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:33.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:44:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:44:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:44:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:33] "GET /metrics HTTP/1.1" 200 228400 "" "Prometheus/2.43.0" 2024-09-10T18:44:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:44:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:44:34.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:34.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:44:34.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:34.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/862669254' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:44:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]: dispatch 2024-09-10T18:44:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:35.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:44:35.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:35.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:35.329 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[29369]: pgmap v395: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:35.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:44:35.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.330 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:35.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:35.331 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:35 smithi131 ceph-mon[40885]: pgmap v395: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:44:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.57071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi131-80207-56"}]': finished 2024-09-10T18:44:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:44:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:44:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:35.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:35.429 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:35.430 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:35 smithi175 ceph-mon[33399]: pgmap v395: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:36.276 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-10T18:44:36.276 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: ok, hit_set contains 192:602f83fe:::foo:head 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9016 ms) 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg_num = 32 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 0 ls 1725993877,0 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 1 ls 1725993877,0 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 2 ls 1725993877,0 2024-09-10T18:44:36.277 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 3 ls 1725993877,0 2024-09-10T18:44:36.278 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 4 ls 1725993877,0 2024-09-10T18:44:36.278 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 5 ls 1725993877,0 2024-09-10T18:44:36.278 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 6 ls 1725993877,0 2024-09-10T18:44:36.278 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 7 ls 1725993877,0 2024-09-10T18:44:36.278 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 8 ls 1725993877,0 2024-09-10T18:44:36.278 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 9 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 10 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 11 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 12 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 13 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 14 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 15 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 16 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 17 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 18 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 19 ls 1725993877,0 2024-09-10T18:44:36.279 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 20 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 21 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 22 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 23 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 24 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 25 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 26 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 27 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 28 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 29 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 30 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg 31 ls 1725993877,0 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: pg_num = 32 2024-09-10T18:44:36.280 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6cac518f:::0:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:02547ec2:::1:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f905c69b:::2:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:cfc208b3:::3:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d83876eb:::4:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b29083e3:::5:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c4fdafeb:::6:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:5c6b0b28:::7:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:bd63b0f1:::8:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e960b815:::9:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:52ea6a34:::10:head 2024-09-10T18:44:36.281 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:89d3ae78:::11:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:de5d7c5f:::12:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:566253c9:::13:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:62a1935d:::14:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:863748b0:::15:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3958e169:::16:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4d4dabf9:::17:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:8391935d:::18:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:28883081:::19:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:69259c59:::20:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4bdb80b7:::21:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a11c5d71:::22:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:271af37b:::23:head 2024-09-10T18:44:36.282 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:95b121be:::24:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:58d1031b:::25:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:0a050783:::26:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c709704c:::27:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:cbe56eaf:::28:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:86b4b162:::29:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:70d89383:::30:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:dd450c7c:::31:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6d5729b1:::32:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c388f3fb:::33:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:56cfea31:::34:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9dbc1bf7:::35:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:40b74ccd:::36:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4d5aaf42:::37:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:920f362c:::38:head 2024-09-10T18:44:36.283 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6cc53222:::39:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9cad833f:::40:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:1ea84d41:::41:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c4480ef6:::42:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a694361e:::43:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d1bd33e9:::44:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ddc2cd5d:::45:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:2b782207:::46:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7b187fca:::47:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:90ecdf6f:::48:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a5ed95fe:::49:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ea0eaa55:::50:head 2024-09-10T18:44:36.284 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f33ef17b:::51:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a0d1b2f6:::52:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:60c5229e:::53:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:edcbc575:::54:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:102cf253:::55:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:efb7fb0b:::56:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:50d0a326:::57:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d4dc5daf:::58:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3a130462:::59:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ec87ed71:::60:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d5bc9454:::61:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3ddfe313:::62:head 2024-09-10T18:44:36.285 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7c2816b9:::63:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:47e00e4d:::64:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c6410c18:::65:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b48ed237:::66:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:cd63ad31:::67:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b179e92b:::68:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:0d9f741a:::69:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6d3352ae:::70:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c6d5c19e:::71:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:bc4729c3:::72:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:77e930b9:::73:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:0abeecfd:::74:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b7c37e15:::75:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b6378398:::76:head 2024-09-10T18:44:36.286 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:02bd68de:::77:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:cc795d2d:::78:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:630d4fea:::79:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e0d29ef5:::80:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:fd6f13d2:::81:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:606461d5:::82:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:eadbdc43:::83:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:8761d0bb:::84:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9ef0186f:::85:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e0d41294:::86:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:961de695:::87:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:1423148f:::88:head 2024-09-10T18:44:36.287 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:633a8fa2:::89:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a8653809:::90:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3dac8b33:::91:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:35aad435:::92:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f6dcc343:::93:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:dbbdad87:::94:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:1cb48ce0:::95:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:03cd461c:::96:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:17a4ea99:::97:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9993c9a7:::98:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6394211c:::99:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:94c7ae57:::100:head 2024-09-10T18:44:36.288 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6fdee5bb:::101:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9a477fd1:::102:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:eb850916:::103:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:affc56b9:::104:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b42dc814:::105:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f319f8f0:::106:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9a40b9de:::107:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:8b524f28:::108:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e3de589f:::109:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:90f90a5b:::110:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a7b4f1d7:::111:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:af51766e:::112:head 2024-09-10T18:44:36.289 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b6f90bd1:::113:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e0261208:::114:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c9569ef7:::115:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:61bebe50:::116:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:fe93412b:::117:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d3d38bee:::118:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3100ba0c:::119:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d0560ada:::120:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f0ea8b35:::121:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:766f231a:::122:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a07a2582:::123:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:bd7c6b3a:::124:head 2024-09-10T18:44:36.290 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:fb2ddaff:::125:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4408e1fe:::126:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ee1df7a7:::127:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c3002909:::128:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4f48ffa9:::129:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:edf38733:::130:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c08425c0:::131:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:5f902d98:::132:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:41ea2c93:::133:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:813cee13:::134:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:0131818d:::135:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:26ba5a85:::136:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:381b8a5a:::137:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:28797e47:::138:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:bfca7f22:::139:head 2024-09-10T18:44:36.291 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:36807075:::140:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:80b03975:::141:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:5c15709b:::142:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f39ea15e:::143:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ea992956:::144:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:48887b1c:::145:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:9f24a9dd:::146:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:987f100b:::147:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d2dd3581:::148:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7fed1808:::149:head 2024-09-10T18:44:36.292 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c80b70e9:::150:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:85ed90f9:::151:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:36428b24:::152:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d044c34a:::153:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7c18bf58:::154:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d1c21232:::155:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a7a3c575:::156:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:87da0633:::157:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d5ac3822:::158:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3f20522d:::159:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6ca26563:::160:head 2024-09-10T18:44:36.293 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:532ce135:::161:head 2024-09-10T18:44:36.294 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:c78863e6:::162:head 2024-09-10T18:44:36.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:44:36.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:36.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:44:36.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:44:36.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:44:36.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:36.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:36 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi131-80207-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:36.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:36 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:44:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-80444-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:44:37.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: pgmap v398: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]: dispatch 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]: dispatch 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]': finished 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[29369]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:44:37.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-80444-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: pgmap v398: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]: dispatch 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]: dispatch 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]': finished 2024-09-10T18:44:37.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:37 smithi131 ceph-mon[40885]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi131-80444-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: pgmap v398: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]: dispatch 2024-09-10T18:44:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]: dispatch 2024-09-10T18:44:37.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi131-80207-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:37.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-43"}]': finished 2024-09-10T18:44:37.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:37 smithi175 ceph-mon[33399]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:44:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:39 smithi131 ceph-mon[29369]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:44:39.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:39 smithi131 ceph-mon[29369]: pgmap v401: 204 pgs: 4 creating+activating, 200 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:39.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:39 smithi131 ceph-mon[40885]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:44:39.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:39 smithi131 ceph-mon[40885]: pgmap v401: 204 pgs: 4 creating+activating, 200 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:39 smithi175 ceph-mon[33399]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:44:39.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:39 smithi175 ceph-mon[33399]: pgmap v401: 204 pgs: 4 creating+activating, 200 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[29369]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:44:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:40.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[40885]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:44:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:40.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:40 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:40 smithi175 ceph-mon[33399]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:44:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:40 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:40.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[29369]: pgmap v404: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:41.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:41 smithi131 ceph-mon[40885]: pgmap v404: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1581640685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]: dispatch 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:44:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:41 smithi175 ceph-mon[33399]: pgmap v404: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:42.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:42 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.57077 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi131-80207-57"}]': finished 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:42.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:42 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[29369]: pgmap v407: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:43.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-mon[40885]: pgmap v407: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi131-80207-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-mon[33399]: pgmap v407: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:43.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:44:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:44:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:44:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:43] "GET /metrics HTTP/1.1" 200 228400 "" "Prometheus/2.43.0" 2024-09-10T18:44:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:44 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:44:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:44:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:44 smithi175 ceph-mon[33399]: osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:44:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:44 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:44:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[29369]: osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:44:44.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:44:44.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[40885]: osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:44:44.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:44:44.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:45.329 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:45.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:45.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:44:45.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:44:45.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:44:45.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:44:45.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: pgmap v410: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:45.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:44:45.330 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:45 smithi175 ceph-mon[33399]: osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: pgmap v410: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:44:45.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[29369]: osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi131-80207-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: pgmap v410: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:44:45.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:45 smithi131 ceph-mon[40885]: osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:44:46.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:46 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:46.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:46 smithi175 ceph-mon[33399]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:44:46.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:46 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:46.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:46 smithi131 ceph-mon[29369]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:44:46.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:46 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:46.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:46 smithi131 ceph-mon[40885]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:44:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:47 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:47 smithi175 ceph-mon[33399]: pgmap v413: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:47 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:47.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:47 smithi175 ceph-mon[33399]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:44:47.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:47.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:47.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[29369]: pgmap v413: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:47.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:47.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[29369]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:44:47.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:47.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:47.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[40885]: pgmap v413: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:47.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:47.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:47 smithi131 ceph-mon[40885]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:44:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:48 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:48 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:48 smithi175 ceph-mon[33399]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:48.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:48 smithi175 ceph-mon[33399]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:44:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[29369]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:48.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[29369]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:44:48.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3118596390' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]: dispatch 2024-09-10T18:44:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:44:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[40885]: from='client.57083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi131-80207-58"}]': finished 2024-09-10T18:44:48.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:48 smithi131 ceph-mon[40885]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:44:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:49.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:49 smithi175 ceph-mon[33399]: pgmap v416: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:49.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[29369]: pgmap v416: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:49.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:49 smithi131 ceph-mon[40885]: pgmap v416: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:50 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:50 smithi175 ceph-mon[33399]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:44:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:50 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[29369]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[40885]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:44:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:50 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:51 smithi175 ceph-mon[33399]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:51 smithi175 ceph-mon[33399]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:44:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:51 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:51 smithi175 ceph-mon[33399]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[29369]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[29369]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[29369]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[40885]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[40885]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:44:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi131-80207-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:51.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:51 smithi131 ceph-mon[40885]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:44:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:52 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:52 smithi175 ceph-mon[33399]: osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:44:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:52 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:52 smithi131 ceph-mon[29369]: osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:44:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:52 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:52 smithi131 ceph-mon[40885]: osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:44:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:53 smithi175 ceph-mon[33399]: pgmap v422: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:53 smithi175 ceph-mon[33399]: osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:44:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:53 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:53.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:44:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:44:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[40885]: pgmap v422: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[40885]: osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:44:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:53.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[29369]: pgmap v422: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:44:53.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[29369]: osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:44:53.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:53.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:44:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:53] "GET /metrics HTTP/1.1" 200 228379 "" "Prometheus/2.43.0" 2024-09-10T18:44:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:54 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:54 smithi175 ceph-mon[33399]: osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:44:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:54 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[29369]: osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:44:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:44:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[40885]: osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:44:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3375540583' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:54 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]: dispatch 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7462ddf6:::1.RoundTripAppendPP (2721 ms) 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3013 ms) 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3018 ms) 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-10T18:44:55.137 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3010 ms) 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: setting pool EIO 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: max_success 1429, min_failed 1430 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4007 ms) 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3016 ms) 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (120940 ms total) 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3025 ms) 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-10T18:44:55.138 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9025 ms) 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14920 ms) 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3246 ms) 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30217 ms total) 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (2765 ms) 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (2765 ms total) 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-09-10T18:44:55.139 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13987 ms) 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6996 ms) 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7013 ms) 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7012 ms) 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3032 ms) 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7038 ms) 2024-09-10T18:44:55.140 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6912 ms) 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7033 ms) 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7020 ms) 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7024 ms) 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7036 ms) 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7030 ms) 2024-09-10T18:44:55.141 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-10T18:44:55.142 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7045 ms) 2024-09-10T18:44:55.391 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:44:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:44:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:44:55.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:55 smithi175 ceph-mon[33399]: pgmap v425: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:44:55.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:55 smithi175 ceph-mon[33399]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:55.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:55 smithi175 ceph-mon[33399]: osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:44:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:55 smithi131 ceph-mon[29369]: pgmap v425: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:44:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:55 smithi131 ceph-mon[29369]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:55 smithi131 ceph-mon[29369]: osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:44:55.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:55 smithi131 ceph-mon[40885]: pgmap v425: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:44:55.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:55 smithi131 ceph-mon[40885]: from='client.57086 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi131-80207-59"}]': finished 2024-09-10T18:44:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:55 smithi131 ceph-mon[40885]: osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:44:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:44:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/20269422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: pgmap v428: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:44:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:57.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]: dispatch 2024-09-10T18:44:57.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]: dispatch 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/20269422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: pgmap v428: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]: dispatch 2024-09-10T18:44:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]: dispatch 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/20269422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: pgmap v428: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]: dispatch 2024-09-10T18:44:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]: dispatch 2024-09-10T18:44:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:58 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/20269422' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]': finished 2024-09-10T18:44:58.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:58 smithi175 ceph-mon[33399]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:44:58.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:58.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/20269422' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:58.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]': finished 2024-09-10T18:44:58.625 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[29369]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:44:58.626 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:44:58.626 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/20269422' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi131-80207-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:44:58.626 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-45"}]': finished 2024-09-10T18:44:58.626 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:58 smithi131 ceph-mon[40885]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:44:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:44:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: pgmap v431: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:59.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:44:59 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:59.478 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:44:59.478 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.478 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: pgmap v431: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: pgmap v431: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.479 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:44:59.480 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:44:59.480 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:44:59 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:00.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:00 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:00.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:00 smithi175 ceph-mon[33399]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:45:00.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:00.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:00 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:00.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:00.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[29369]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:45:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi131-80207-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[40885]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:45:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:00 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: pgmap v434: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:01.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:45:01.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:01.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: pgmap v434: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:45:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: pgmap v434: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi131-80207-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:01.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:45:01.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:01.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]': finished 2024-09-10T18:45:03.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-mon[33399]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:45:03.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]: dispatch 2024-09-10T18:45:03.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]: dispatch 2024-09-10T18:45:03.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:03.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-mon[33399]: pgmap v437: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]': finished 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[29369]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]: dispatch 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]: dispatch 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[29369]: pgmap v437: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:03.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-47"}]': finished 2024-09-10T18:45:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[40885]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:45:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]: dispatch 2024-09-10T18:45:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]: dispatch 2024-09-10T18:45:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-mon[40885]: pgmap v437: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:03.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:45:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:45:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:45:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:03] "GET /metrics HTTP/1.1" 200 230816 "" "Prometheus/2.43.0" 2024-09-10T18:45:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]': finished 2024-09-10T18:45:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:45:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]': finished 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:04.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-47", "mode": "writeback"}]': finished 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:04.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:45:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:45:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:05 smithi175 ceph-mon[33399]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[29369]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3673361666' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]: dispatch 2024-09-10T18:45:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:05 smithi131 ceph-mon[40885]: pgmap v440: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:06.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:06.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:45:06.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:06.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:45:06.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:06.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:06.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:06.580 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:06 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:45:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.57098 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi131-80207-61"}]': finished 2024-09-10T18:45:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T18:45:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:06.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:06 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:07.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:07.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:45:07.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:07.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:45:07.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[40885]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:07.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:07.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:07.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:07 smithi131 ceph-mon[29369]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:07 smithi175 ceph-mon[33399]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:45:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:45:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:45:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:45:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:45:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:45:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:08.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:45:08.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:45:08.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:45:08.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:45:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:45:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi131-80207-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:45:08.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:45:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[29369]: pgmap v446: 236 pgs: 6 creating+peering, 2 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:45:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[29369]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:45:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[40885]: pgmap v446: 236 pgs: 6 creating+peering, 2 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:45:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[40885]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:45:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:09 smithi175 ceph-mon[33399]: pgmap v446: 236 pgs: 6 creating+peering, 2 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:09 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:45:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:09 smithi175 ceph-mon[33399]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T18:45:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:45:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[29369]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:11 smithi131 ceph-mon[40885]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:11 smithi175 ceph-mon[33399]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]': finished 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]': finished 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:12.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]': finished 2024-09-10T18:45:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T18:45:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1289180909' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]: dispatch 2024-09-10T18:45:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:12.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:12.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:12.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-47"}]: dispatch 2024-09-10T18:45:13.541 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:13.541 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:45:13.541 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: pgmap v452: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:13.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:13.542 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: pgmap v452: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.57104 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi131-80207-62"}]': finished 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: pgmap v452: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:13.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:45:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:45:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:45:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:13] "GET /metrics HTTP/1.1" 200 230816 "" "Prometheus/2.43.0" 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:14.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi131-80207-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: pgmap v455: 228 pgs: 11 creating+peering, 21 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: pgmap v455: 228 pgs: 11 creating+peering, 21 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:45:15.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:15.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: pgmap v455: 228 pgs: 11 creating+peering, 21 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi131-80207-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T18:45:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:15.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]': finished 2024-09-10T18:45:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[29369]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:45:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:45:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:45:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[29369]: pgmap v458: 236 pgs: 11 creating+peering, 29 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:45:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]': finished 2024-09-10T18:45:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[40885]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:45:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:45:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:45:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[40885]: pgmap v458: 236 pgs: 11 creating+peering, 29 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:45:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:17 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:17 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-49"}]': finished 2024-09-10T18:45:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:17 smithi175 ceph-mon[33399]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T18:45:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:45:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:17 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]: dispatch 2024-09-10T18:45:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:17 smithi175 ceph-mon[33399]: pgmap v458: 236 pgs: 11 creating+peering, 29 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:45:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:17 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]': finished 2024-09-10T18:45:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:45:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]': finished 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:18.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-49", "mode": "readproxy"}]': finished 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[29369]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:45:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:19.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[29369]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:45:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[40885]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:45:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:19.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:19 smithi131 ceph-mon[40885]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:45:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:19 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:19 smithi175 ceph-mon[33399]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T18:45:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1652600252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:19 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]: dispatch 2024-09-10T18:45:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:19 smithi175 ceph-mon[33399]: pgmap v461: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:20.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:20 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.57110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi131-80207-63"}]': finished 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:20 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[29369]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[29369]: pgmap v464: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[40885]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:45:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:21 smithi131 ceph-mon[40885]: pgmap v464: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:45:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:21 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi131-80207-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:21 smithi175 ceph-mon[33399]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T18:45:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:21 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:21 smithi175 ceph-mon[33399]: pgmap v464: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:45:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:22 smithi131 ceph-mon[29369]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:45:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:22 smithi131 ceph-mon[40885]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:45:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:22 smithi175 ceph-mon[33399]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T18:45:23.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:23 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:23.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:23 smithi175 ceph-mon[33399]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:45:23.552 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:23 smithi175 ceph-mon[33399]: pgmap v467: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-mon[29369]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:45:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-mon[29369]: pgmap v467: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi131-80207-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-mon[40885]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T18:45:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-mon[40885]: pgmap v467: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:23.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:45:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:45:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:45:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:23] "GET /metrics HTTP/1.1" 200 228405 "" "Prometheus/2.43.0" 2024-09-10T18:45:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:24 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:24 smithi131 ceph-mon[40885]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:45:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:24 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:24 smithi131 ceph-mon[29369]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:45:24.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:24 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:24 smithi175 ceph-mon[33399]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T18:45:24.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-mon[33399]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:45:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-mon[33399]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:25 smithi175 ceph-mon[33399]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[29369]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[29369]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[29369]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[40885]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T18:45:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[40885]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:25 smithi131 ceph-mon[40885]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[29369]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[29369]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[40885]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:26 smithi131 ceph-mon[40885]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:45:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2346769395' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:26 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]: dispatch 2024-09-10T18:45:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:26 smithi175 ceph-mon[33399]: from='client.57116 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi131-80207-64"}]': finished 2024-09-10T18:45:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:26 smithi175 ceph-mon[33399]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T18:45:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: pgmap v473: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: pgmap v473: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:27.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:27 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: pgmap v473: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:27 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[29369]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:45:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:28.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:28.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[40885]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:45:28.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:28.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:28.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:28.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:28 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi131-80207-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:28 smithi175 ceph-mon[33399]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T18:45:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:28 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[29369]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[29369]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[40885]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:29.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[40885]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:45:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:29 smithi175 ceph-mon[33399]: pgmap v475: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:29 smithi175 ceph-mon[33399]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T18:45:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullP63:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:5d165639:::164:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f43765fc:::165:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b4c720e9:::166:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e694b040:::167:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:afa38db2:::168:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:77ba9f53:::169:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:87495034:::170:head 2024-09-10T18:45:30.320 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7c96bf0e:::171:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:dbe346cc:::172:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:e943ec24:::173:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f97a9c0c:::174:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:6f26e74d:::175:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4f95e106:::176:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:0e6f2f8f:::177:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:05db05f1:::178:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:38a78d66:::179:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:d095610b:::180:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:a1a9d709:::181:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:1e5d39db:::182:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:f7df4fb9:::183:head 2024-09-10T18:45:30.321 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:03a7f161:::184:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ba70721e:::185:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:28e5662d:::186:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:973d52de:::187:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:4303eb1c:::188:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:b990b48e:::189:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:29b8165b:::190:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:3547f197:::191:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:7e260936:::192:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:1abec7b1:::193:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:10fdda93:::194:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:15817eea:::195:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:770bab57:::196:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:ed9e13e7:::197:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:71471a8f:::198:head 2024-09-10T18:45:30.322 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: checking for 195:10fb1d02:::199:head 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8034 ms) 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first is 1725993886 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,1725993891,1725993892,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,1725993891,1725993892,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,1725993891,1725993892,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,1725993891,1725993892,1725993894,1725993895,0 2024-09-10T18:45:30.323 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,1725993891,1725993892,1725993894,1725993895,0 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993886,1725993888,1725993889,1725993891,1725993892,1725993894,1725993895,0 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725993889,1725993891,1725993892,1725993894,1725993895,1725993897,1725993898,0 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first now 1725993889, trimmed 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20109 ms) 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: foo0 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14057 ms) 2024-09-10T18:45:30.324 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-10T18:45:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]': finished 2024-09-10T18:45:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:45:30.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]': finished 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:30.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:30 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi131-80207-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]': finished 2024-09-10T18:45:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T18:45:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:30.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:30.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:30.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:30 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-49"}]: dispatch 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[29369]: pgmap v478: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[29369]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[40885]: pgmap v478: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[40885]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:45:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:31 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:31 smithi175 ceph-mon[33399]: pgmap v478: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:45:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:31 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:31 smithi175 ceph-mon[33399]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T18:45:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:31 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:45:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:45:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:32.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:32.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:32.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:32.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:32.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:45:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:32.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:32.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:33.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:45:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:33.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:33.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2980288815' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]: dispatch 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.57122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi131-80207-65"}]': finished 2024-09-10T18:45:33.806 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:45:33.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T18:45:33.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:33.807 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:45:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:33] "GET /metrics HTTP/1.1" 200 228413 "" "Prometheus/2.43.0" 2024-09-10T18:45:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-80207-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]': finished 2024-09-10T18:45:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-80207-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:45:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]: dispatch 2024-09-10T18:45:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-80207-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]: dispatch 2024-09-10T18:45:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-80207-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]': finished 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-80207-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-80207-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-80207-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:45:34.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:34.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-51"}]': finished 2024-09-10T18:45:34.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi131-80207-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:45:34.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T18:45:34.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]: dispatch 2024-09-10T18:45:34.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-80207-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:34.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]: dispatch 2024-09-10T18:45:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:45:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:35 smithi131 ceph-mon[29369]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:45:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:35 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:35 smithi131 ceph-mon[40885]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:45:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:35 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:35 smithi175 ceph-mon[33399]: pgmap v484: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:45:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:35 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]': finished 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-80207-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:45:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]': finished 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-80207-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:36.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-51", "mode": "writeback"}]': finished 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi131-80207-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:45:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T18:45:36.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:36.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:45:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[29369]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:45:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:45:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[29369]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:45:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[40885]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:45:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:45:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[40885]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:45:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:37 smithi175 ceph-mon[33399]: pgmap v487: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:45:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:37 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:45:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:37 smithi175 ceph-mon[33399]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T18:45:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:45:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:38 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:45:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:38 smithi175 ceph-mon[33399]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:45:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:45:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[29369]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:45:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:45:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:45:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[40885]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T18:45:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: pgmap v490: 236 pgs: 236 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:45:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: pgmap v490: 236 pgs: 236 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:45:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: pgmap v490: 236 pgs: 236 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]: dispatch 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:45:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:45:40.344 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ P2 2024-09-10T18:45:40.344 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3014 ms) 2024-09-10T18:45:40.344 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-10T18:45:40.344 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7037 ms) 2024-09-10T18:45:40.344 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-10T18:45:40.344 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7027 ms) 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7026 ms) 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7045 ms) 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7041 ms) 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7015 ms) 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139385 ms total) 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-10T18:45:40.345 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (293307 ms total) 2024-09-10T18:45:40.346 INFO:tasks.workunit.client.0.smithi131.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-10T18:45:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:41 smithi175 ceph-mon[33399]: pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:45:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:41 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:45:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:41 smithi175 ceph-mon[33399]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[29369]: pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[29369]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[40885]: pgmap v493: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1779705132' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi131-80207-66"}]': finished 2024-09-10T18:45:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:45:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:41 smithi131 ceph-mon[40885]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T18:45:42.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:42 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:45:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:42 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:45:42.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:42 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:45:43.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:43 smithi175 ceph-mon[33399]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:45:43.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:45:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:45:43.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:43 smithi131 ceph-mon[29369]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:45:43.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:43 smithi131 ceph-mon[40885]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:45:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:45:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:43] "GET /metrics HTTP/1.1" 200 228413 "" "Prometheus/2.43.0" 2024-09-10T18:45:44.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:44.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:44.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:45.395 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:45:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:45.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:45 smithi175 ceph-mon[33399]: pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:45:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:45 smithi131 ceph-mon[29369]: pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:45:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:45 smithi131 ceph-mon[40885]: pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:45:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:47 smithi175 ceph-mon[33399]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 642 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:45:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:47 smithi131 ceph-mon[29369]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 642 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:45:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:47 smithi131 ceph-mon[40885]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 642 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:45:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:45:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:45:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:49 smithi175 ceph-mon[33399]: pgmap v498: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:45:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:49 smithi131 ceph-mon[29369]: pgmap v498: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:45:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:49 smithi131 ceph-mon[40885]: pgmap v498: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:45:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:51 smithi175 ceph-mon[33399]: pgmap v499: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:45:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:51 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:51 smithi175 ceph-mon[33399]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:45:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:51.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:51 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[29369]: pgmap v499: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:45:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:51.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[29369]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:45:51.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:51.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:51.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[40885]: pgmap v499: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:45:51.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:45:51.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[40885]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T18:45:51.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:51.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:51 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]': finished 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[29369]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:52.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]': finished 2024-09-10T18:45:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[40885]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:45:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]': finished 2024-09-10T18:45:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:52 smithi175 ceph-mon[33399]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T18:45:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-51"}]: dispatch 2024-09-10T18:45:53.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-mon[29369]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:45:53.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:45:53.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-mon[29369]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:45:53.804 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-mon[40885]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:45:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:45:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-mon[40885]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:45:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:53 smithi175 ceph-mon[33399]: pgmap v502: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:45:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:53 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:45:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:53 smithi175 ceph-mon[33399]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T18:45:53.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:45:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:45:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:45:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:53] "GET /metrics HTTP/1.1" 200 228521 "" "Prometheus/2.43.0" 2024-09-10T18:45:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[29369]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:45:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[40885]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:45:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:54.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:54.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:54 smithi175 ceph-mon[33399]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T18:45:54.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:54.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:54 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:54.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:45:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:45:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-53"}]: dispatch 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-53"}]: dispatch 2024-09-10T18:45:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[29369]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-53"}]: dispatch 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-53"}]: dispatch 2024-09-10T18:45:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:55 smithi131 ceph-mon[40885]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:45:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: pgmap v505: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:45:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T18:45:56.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:45:56.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:56.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:56.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-53"}]: dispatch 2024-09-10T18:45:56.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-53"}]: dispatch 2024-09-10T18:45:56.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:55 smithi175 ceph-mon[33399]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T18:45:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[29369]: pgmap v508: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:45:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[29369]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:45:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[40885]: pgmap v508: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:45:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[40885]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:45:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:57 smithi175 ceph-mon[33399]: pgmap v508: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:45:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:57 smithi175 ceph-mon[33399]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T18:45:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-55"}]: dispatch 2024-09-10T18:45:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-55"}]: dispatch 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-55"}]: dispatch 2024-09-10T18:45:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-55"}]: dispatch 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-55"}]: dispatch 2024-09-10T18:45:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-55"}]: dispatch 2024-09-10T18:45:59.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:59 smithi131 ceph-mon[29369]: pgmap v511: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-10T18:45:59.728 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:45:59 smithi131 ceph-mon[29369]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:45:59.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:59 smithi131 ceph-mon[40885]: pgmap v511: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-10T18:45:59.729 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:45:59 smithi131 ceph-mon[40885]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:45:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:59 smithi175 ceph-mon[33399]: pgmap v511: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-09-10T18:45:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:45:59 smithi175 ceph-mon[33399]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T18:46:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:00 smithi131 ceph-mon[29369]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:46:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:00 smithi131 ceph-mon[40885]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:46:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:00 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:00 smithi175 ceph-mon[33399]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T18:46:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:46:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:01.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:01.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:46:01.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:01.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:01.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-57"}]: dispatch 2024-09-10T18:46:01.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-57"}]: dispatch 2024-09-10T18:46:01.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:46:01.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:01.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:01.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:46:01.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:01.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:01.830 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-57"}]: dispatch 2024-09-10T18:46:01.831 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-57"}]: dispatch 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: pgmap v514: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-57"}]: dispatch 2024-09-10T18:46:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-57"}]: dispatch 2024-09-10T18:46:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:02 smithi131 ceph-mon[29369]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:46:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:46:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:02.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:02 smithi131 ceph-mon[40885]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:46:02.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:46:02.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:02 smithi175 ceph-mon[33399]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T18:46:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:46:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:03.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[29369]: pgmap v517: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[29369]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[40885]: pgmap v517: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[40885]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:03 smithi175 ceph-mon[33399]: pgmap v517: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:03 smithi175 ceph-mon[33399]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T18:46:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:03.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:46:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:46:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:46:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:03] "GET /metrics HTTP/1.1" 200 226050 "" "Prometheus/2.43.0" 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-59"}]: dispatch 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-59"}]: dispatch 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[40885]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-59"}]: dispatch 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-59"}]: dispatch 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:04 smithi131 ceph-mon[29369]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-59"}]: dispatch 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-59"}]: dispatch 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:05.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:04 smithi175 ceph-mon[33399]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T18:46:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:46:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[29369]: pgmap v520: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:46:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[29369]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:46:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[40885]: pgmap v520: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:46:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[40885]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:46:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:05 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:05 smithi175 ceph-mon[33399]: pgmap v520: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:46:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:05 smithi175 ceph-mon[33399]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T18:46:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:05 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:46:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:46:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:07.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:07.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:07 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:07 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[29369]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:46:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-61"}]: dispatch 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-61"}]: dispatch 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[40885]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-61"}]: dispatch 2024-09-10T18:46:08.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-61"}]: dispatch 2024-09-10T18:46:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:08 smithi175 ceph-mon[33399]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T18:46:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-61"}]: dispatch 2024-09-10T18:46:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-61"}]: dispatch 2024-09-10T18:46:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:09 smithi131 ceph-mon[29369]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:46:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:09 smithi131 ceph-mon[29369]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:46:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:09 smithi131 ceph-mon[40885]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:46:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:09 smithi131 ceph-mon[40885]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:46:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:09 smithi175 ceph-mon[33399]: pgmap v526: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:46:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:09 smithi175 ceph-mon[33399]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T18:46:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[29369]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:46:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[29369]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:46:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[40885]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:46:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:10 smithi131 ceph-mon[40885]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:46:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:10 smithi175 ceph-mon[33399]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T18:46:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:10 smithi175 ceph-mon[33399]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T18:46:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[29369]: pgmap v529: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:46:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[29369]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[40885]: pgmap v529: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:11 smithi131 ceph-mon[40885]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:46:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:11 smithi175 ceph-mon[33399]: pgmap v529: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:46:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:11 smithi175 ceph-mon[33399]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T18:46:12.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:12 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:12.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:12 smithi131 ceph-mon[29369]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:46:12.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:12 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:12.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:12 smithi131 ceph-mon[40885]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:46:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:12 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:46:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:12 smithi175 ceph-mon[33399]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T18:46:13.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:46:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:13] "GET /metrics HTTP/1.1" 200 226050 "" "Prometheus/2.43.0" 2024-09-10T18:46:13.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:13 smithi131 ceph-mon[40885]: pgmap v532: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:13.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:13 smithi131 ceph-mon[40885]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:46:13.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:13 smithi131 ceph-mon[29369]: pgmap v532: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:13.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:13 smithi131 ceph-mon[29369]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:46:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:13 smithi175 ceph-mon[33399]: pgmap v532: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:13 smithi175 ceph-mon[33399]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T18:46:13.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:46:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:46:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:14.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:14 smithi131 ceph-mon[29369]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:46:14.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:14.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:14 smithi131 ceph-mon[40885]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:46:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:14 smithi175 ceph-mon[33399]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T18:46:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:46:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:15 smithi131 ceph-mon[29369]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:46:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:15 smithi131 ceph-mon[29369]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:46:15.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:15 smithi131 ceph-mon[40885]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:46:15.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:15 smithi131 ceph-mon[40885]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:46:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:15 smithi175 ceph-mon[33399]: pgmap v535: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:46:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:15 smithi175 ceph-mon[33399]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T18:46:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:17 smithi131 ceph-mon[29369]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:46:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:17 smithi131 ceph-mon[40885]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:46:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:17 smithi175 ceph-mon[33399]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-09-10T18:46:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:19 smithi175 ceph-mon[33399]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-10T18:46:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:19 smithi131 ceph-mon[29369]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-10T18:46:20.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:19 smithi131 ceph-mon[40885]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.9 KiB/s wr, 6 op/s 2024-09-10T18:46:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:21 smithi175 ceph-mon[33399]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 603 B/s wr, 2 op/s 2024-09-10T18:46:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:21 smithi131 ceph-mon[29369]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 603 B/s wr, 2 op/s 2024-09-10T18:46:22.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:21 smithi131 ceph-mon[40885]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 603 B/s wr, 2 op/s 2024-09-10T18:46:23.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:22 smithi131 ceph-mon[29369]: osdmap e380: 8 total, 8 up, 8 in 2024-09-10T18:46:23.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:22 smithi131 ceph-mon[29369]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 525 B/s wr, 2 op/s 2024-09-10T18:46:23.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:22 smithi131 ceph-mon[40885]: osdmap e380: 8 total, 8 up, 8 in 2024-09-10T18:46:23.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:22 smithi131 ceph-mon[40885]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 525 B/s wr, 2 op/s 2024-09-10T18:46:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:22 smithi175 ceph-mon[33399]: osdmap e380: 8 total, 8 up, 8 in 2024-09-10T18:46:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:22 smithi175 ceph-mon[33399]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 525 B/s wr, 2 op/s 2024-09-10T18:46:23.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:46:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:46:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:23 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:46:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:23] "GET /metrics HTTP/1.1" 200 228507 "" "Prometheus/2.43.0" 2024-09-10T18:46:24.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:23 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:23 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:24 smithi131 ceph-mon[29369]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 466 B/s wr, 2 op/s 2024-09-10T18:46:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:24 smithi131 ceph-mon[40885]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 466 B/s wr, 2 op/s 2024-09-10T18:46:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:24 smithi175 ceph-mon[33399]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 466 B/s wr, 2 op/s 2024-09-10T18:46:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:46:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:26 smithi131 ceph-mon[29369]: osdmap e381: 8 total, 8 up, 8 in 2024-09-10T18:46:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:26 smithi131 ceph-mon[29369]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:26 smithi131 ceph-mon[40885]: osdmap e381: 8 total, 8 up, 8 in 2024-09-10T18:46:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:26 smithi131 ceph-mon[40885]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:26 smithi175 ceph-mon[33399]: osdmap e381: 8 total, 8 up, 8 in 2024-09-10T18:46:27.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:26 smithi175 ceph-mon[33399]: pgmap v545: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:46:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:29 smithi175 ceph-mon[33399]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:29.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:29 smithi131 ceph-mon[29369]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:29 smithi131 ceph-mon[40885]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:31 smithi175 ceph-mon[33399]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:31.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:31 smithi131 ceph-mon[29369]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:31 smithi131 ceph-mon[40885]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:46:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:46:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:32 smithi175 ceph-mon[33399]: osdmap e382: 8 total, 8 up, 8 in 2024-09-10T18:46:32.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[29369]: osdmap e382: 8 total, 8 up, 8 in 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:32.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:46:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:46:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:46:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:46:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[40885]: osdmap e382: 8 total, 8 up, 8 in 2024-09-10T18:46:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:33 smithi175 ceph-mon[33399]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:33.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:46:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:46:33.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:33 smithi131 ceph-mon[29369]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:33 smithi131 ceph-mon[40885]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:46:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:33] "GET /metrics HTTP/1.1" 200 228499 "" "Prometheus/2.43.0" 2024-09-10T18:46:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:34.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:35.383 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:46:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:35.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:35 smithi175 ceph-mon[33399]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:35 smithi131 ceph-mon[29369]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:35 smithi131 ceph-mon[40885]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:37 smithi175 ceph-mon[33399]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:37.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:37 smithi175 ceph-mon[33399]: osdmap e383: 8 total, 8 up, 8 in 2024-09-10T18:46:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:37 smithi131 ceph-mon[29369]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:37 smithi131 ceph-mon[29369]: osdmap e383: 8 total, 8 up, 8 in 2024-09-10T18:46:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:37 smithi131 ceph-mon[40885]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:46:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:37 smithi131 ceph-mon[40885]: osdmap e383: 8 total, 8 up, 8 in 2024-09-10T18:46:39.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:39 smithi175 ceph-mon[33399]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:39 smithi131 ceph-mon[29369]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:39 smithi131 ceph-mon[40885]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:41.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:41 smithi175 ceph-mon[33399]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:46:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:41 smithi131 ceph-mon[29369]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:46:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:41 smithi131 ceph-mon[40885]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:46:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:42 smithi131 ceph-mon[29369]: osdmap e384: 8 total, 8 up, 8 in 2024-09-10T18:46:43.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:42 smithi131 ceph-mon[29369]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:46:43.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:42 smithi131 ceph-mon[40885]: osdmap e384: 8 total, 8 up, 8 in 2024-09-10T18:46:43.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:42 smithi131 ceph-mon[40885]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:46:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:42 smithi175 ceph-mon[33399]: osdmap e384: 8 total, 8 up, 8 in 2024-09-10T18:46:43.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:42 smithi175 ceph-mon[33399]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:46:43.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:46:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:46:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:43 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:46:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:43] "GET /metrics HTTP/1.1" 200 228499 "" "Prometheus/2.43.0" 2024-09-10T18:46:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:43 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:44.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:43 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:45.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:44 smithi131 ceph-mon[29369]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:45.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:44 smithi131 ceph-mon[40885]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:45.328 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:44 smithi175 ceph-mon[33399]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:46:45.328 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:46:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:47 smithi175 ceph-mon[33399]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:46:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-63"}]: dispatch 2024-09-10T18:46:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-63"}]: dispatch 2024-09-10T18:46:47.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[40885]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:46:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-63"}]: dispatch 2024-09-10T18:46:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-63"}]: dispatch 2024-09-10T18:46:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:47.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[29369]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:46:47.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:47.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:46:47.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-63"}]: dispatch 2024-09-10T18:46:47.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-63"}]: dispatch 2024-09-10T18:46:47.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:46:48.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:48 smithi175 ceph-mon[33399]: osdmap e385: 8 total, 8 up, 8 in 2024-09-10T18:46:48.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:48 smithi131 ceph-mon[29369]: osdmap e385: 8 total, 8 up, 8 in 2024-09-10T18:46:48.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:48 smithi131 ceph-mon[40885]: osdmap e385: 8 total, 8 up, 8 in 2024-09-10T18:46:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:49 smithi175 ceph-mon[33399]: pgmap v560: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:49 smithi175 ceph-mon[33399]: osdmap e386: 8 total, 8 up, 8 in 2024-09-10T18:46:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:49.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:49 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[29369]: pgmap v560: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[29369]: osdmap e386: 8 total, 8 up, 8 in 2024-09-10T18:46:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[40885]: pgmap v560: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[40885]: osdmap e386: 8 total, 8 up, 8 in 2024-09-10T18:46:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:49.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:49 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:46:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:50 smithi175 ceph-mon[33399]: osdmap e387: 8 total, 8 up, 8 in 2024-09-10T18:46:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:50.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[29369]: osdmap e387: 8 total, 8 up, 8 in 2024-09-10T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:46:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[40885]: osdmap e387: 8 total, 8 up, 8 in 2024-09-10T18:46:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:46:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:46:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:51 smithi175 ceph-mon[33399]: pgmap v563: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:51 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:51.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:51 smithi175 ceph-mon[33399]: osdmap e388: 8 total, 8 up, 8 in 2024-09-10T18:46:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:51 smithi131 ceph-mon[29369]: pgmap v563: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:51 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:51 smithi131 ceph-mon[29369]: osdmap e388: 8 total, 8 up, 8 in 2024-09-10T18:46:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:51 smithi131 ceph-mon[40885]: pgmap v563: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:51 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:46:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:51 smithi131 ceph-mon[40885]: osdmap e388: 8 total, 8 up, 8 in 2024-09-10T18:46:52.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:52 smithi175 ceph-mon[33399]: osdmap e389: 8 total, 8 up, 8 in 2024-09-10T18:46:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:52 smithi131 ceph-mon[29369]: osdmap e389: 8 total, 8 up, 8 in 2024-09-10T18:46:52.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:52 smithi131 ceph-mon[40885]: osdmap e389: 8 total, 8 up, 8 in 2024-09-10T18:46:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:53 smithi175 ceph-mon[33399]: pgmap v566: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:53 smithi175 ceph-mon[33399]: osdmap e390: 8 total, 8 up, 8 in 2024-09-10T18:46:53.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:46:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:46:53.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:53 smithi131 ceph-mon[29369]: pgmap v566: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:53.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:53 smithi131 ceph-mon[29369]: osdmap e390: 8 total, 8 up, 8 in 2024-09-10T18:46:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:53 smithi131 ceph-mon[40885]: pgmap v566: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:46:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:53 smithi131 ceph-mon[40885]: osdmap e390: 8 total, 8 up, 8 in 2024-09-10T18:46:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:46:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:53] "GET /metrics HTTP/1.1" 200 228467 "" "Prometheus/2.43.0" 2024-09-10T18:46:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:54 smithi131 ceph-mon[29369]: osdmap e391: 8 total, 8 up, 8 in 2024-09-10T18:46:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:54 smithi131 ceph-mon[40885]: osdmap e391: 8 total, 8 up, 8 in 2024-09-10T18:46:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:54 smithi175 ceph-mon[33399]: osdmap e391: 8 total, 8 up, 8 in 2024-09-10T18:46:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:46:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:46:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:46:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:46:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:55 smithi131 ceph-mon[29369]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T18:46:55.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:55 smithi131 ceph-mon[40885]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T18:46:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:55 smithi175 ceph-mon[33399]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-09-10T18:46:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:57 smithi131 ceph-mon[29369]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-10T18:46:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:57 smithi131 ceph-mon[40885]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-10T18:46:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:57 smithi175 ceph-mon[33399]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-09-10T18:46:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:46:59 smithi131 ceph-mon[40885]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-10T18:46:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:46:59 smithi131 ceph-mon[29369]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-10T18:46:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:46:59 smithi175 ceph-mon[33399]: pgmap v571: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-09-10T18:47:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:01 smithi131 ceph-mon[40885]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:47:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:01 smithi131 ceph-mon[29369]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:47:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:01 smithi175 ceph-mon[33399]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-10T18:47:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:02 smithi131 ceph-mon[29369]: osdmap e392: 8 total, 8 up, 8 in 2024-09-10T18:47:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:02 smithi131 ceph-mon[29369]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:02 smithi131 ceph-mon[40885]: osdmap e392: 8 total, 8 up, 8 in 2024-09-10T18:47:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:02 smithi131 ceph-mon[40885]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:02 smithi175 ceph-mon[33399]: osdmap e392: 8 total, 8 up, 8 in 2024-09-10T18:47:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:03.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:02 smithi175 ceph-mon[33399]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:47:03.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:47:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:03] "GET /metrics HTTP/1.1" 200 228510 "" "Prometheus/2.43.0" 2024-09-10T18:47:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.5", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.e", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1d", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1e", "id": [7, 4]}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-65"}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-65"}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.5", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.e", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1d", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1e", "id": [7, 4]}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-65"}]: dispatch 2024-09-10T18:47:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-65"}]: dispatch 2024-09-10T18:47:04.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:03 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.5", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.e", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1d", "id": [7, 2]}]: dispatch 2024-09-10T18:47:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1e", "id": [7, 4]}]: dispatch 2024-09-10T18:47:04.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-65"}]: dispatch 2024-09-10T18:47:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-65"}]: dispatch 2024-09-10T18:47:04.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:03 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.5", "id": [7, 2]}]': finished 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.e", "id": [7, 2]}]': finished 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1d", "id": [7, 2]}]': finished 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1e", "id": [7, 4]}]': finished 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[29369]: osdmap e393: 8 total, 8 up, 8 in 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[29369]: pgmap v576: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.5", "id": [7, 2]}]': finished 2024-09-10T18:47:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.e", "id": [7, 2]}]': finished 2024-09-10T18:47:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1d", "id": [7, 2]}]': finished 2024-09-10T18:47:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1e", "id": [7, 4]}]': finished 2024-09-10T18:47:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[40885]: osdmap e393: 8 total, 8 up, 8 in 2024-09-10T18:47:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:04 smithi131 ceph-mon[40885]: pgmap v576: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:47:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:04 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.5", "id": [7, 2]}]': finished 2024-09-10T18:47:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:04 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.e", "id": [7, 2]}]': finished 2024-09-10T18:47:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:04 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1d", "id": [7, 2]}]': finished 2024-09-10T18:47:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:04 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "9.1e", "id": [7, 4]}]': finished 2024-09-10T18:47:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:04 smithi175 ceph-mon[33399]: osdmap e393: 8 total, 8 up, 8 in 2024-09-10T18:47:05.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:04 smithi175 ceph-mon[33399]: pgmap v576: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:06 smithi131 ceph-mon[29369]: osdmap e394: 8 total, 8 up, 8 in 2024-09-10T18:47:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:06 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:06 smithi131 ceph-mon[40885]: osdmap e394: 8 total, 8 up, 8 in 2024-09-10T18:47:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:06 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:06.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:06 smithi175 ceph-mon[33399]: osdmap e394: 8 total, 8 up, 8 in 2024-09-10T18:47:06.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:06.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:06 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[29369]: osdmap e395: 8 total, 8 up, 8 in 2024-09-10T18:47:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[29369]: pgmap v579: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[40885]: osdmap e395: 8 total, 8 up, 8 in 2024-09-10T18:47:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:07.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:07 smithi131 ceph-mon[40885]: pgmap v579: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:07 smithi175 ceph-mon[33399]: osdmap e395: 8 total, 8 up, 8 in 2024-09-10T18:47:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:07.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:07 smithi175 ceph-mon[33399]: pgmap v579: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[29369]: osdmap e396: 8 total, 8 up, 8 in 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-67"}]: dispatch 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-67"}]: dispatch 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[40885]: osdmap e396: 8 total, 8 up, 8 in 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:08.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-67"}]: dispatch 2024-09-10T18:47:08.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-67"}]: dispatch 2024-09-10T18:47:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:08 smithi175 ceph-mon[33399]: osdmap e396: 8 total, 8 up, 8 in 2024-09-10T18:47:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-67"}]: dispatch 2024-09-10T18:47:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-67"}]: dispatch 2024-09-10T18:47:09.326 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:09 smithi131 ceph-mon[40885]: osdmap e397: 8 total, 8 up, 8 in 2024-09-10T18:47:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:09 smithi131 ceph-mon[40885]: pgmap v582: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:47:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:09 smithi131 ceph-mon[29369]: osdmap e397: 8 total, 8 up, 8 in 2024-09-10T18:47:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:09 smithi131 ceph-mon[29369]: pgmap v582: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:47:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:09 smithi175 ceph-mon[33399]: osdmap e397: 8 total, 8 up, 8 in 2024-09-10T18:47:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:09 smithi175 ceph-mon[33399]: pgmap v582: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:47:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:10 smithi131 ceph-mon[29369]: osdmap e398: 8 total, 8 up, 8 in 2024-09-10T18:47:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:10.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:10 smithi131 ceph-mon[40885]: osdmap e398: 8 total, 8 up, 8 in 2024-09-10T18:47:10.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:10.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:10 smithi175 ceph-mon[33399]: osdmap e398: 8 total, 8 up, 8 in 2024-09-10T18:47:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:10.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: osdmap e399: 8 total, 8 up, 8 in 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-69"}]: dispatch 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-69"}]: dispatch 2024-09-10T18:47:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[29369]: pgmap v585: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: osdmap e399: 8 total, 8 up, 8 in 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-69"}]: dispatch 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-69"}]: dispatch 2024-09-10T18:47:11.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:11 smithi131 ceph-mon[40885]: pgmap v585: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:47:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: osdmap e399: 8 total, 8 up, 8 in 2024-09-10T18:47:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:11.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-69"}]: dispatch 2024-09-10T18:47:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-69"}]: dispatch 2024-09-10T18:47:11.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:11 smithi175 ceph-mon[33399]: pgmap v585: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:47:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:12 smithi131 ceph-mon[29369]: osdmap e400: 8 total, 8 up, 8 in 2024-09-10T18:47:12.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:12 smithi131 ceph-mon[40885]: osdmap e400: 8 total, 8 up, 8 in 2024-09-10T18:47:12.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:12 smithi175 ceph-mon[33399]: osdmap e400: 8 total, 8 up, 8 in 2024-09-10T18:47:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[29369]: osdmap e401: 8 total, 8 up, 8 in 2024-09-10T18:47:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[29369]: pgmap v588: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:13.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[40885]: osdmap e401: 8 total, 8 up, 8 in 2024-09-10T18:47:13.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-mon[40885]: pgmap v588: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:13 smithi175 ceph-mon[33399]: osdmap e401: 8 total, 8 up, 8 in 2024-09-10T18:47:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:13 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:13 smithi175 ceph-mon[33399]: pgmap v588: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:13.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:47:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:47:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:13] "GET /metrics HTTP/1.1" 200 228510 "" "Prometheus/2.43.0" 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: osdmap e402: 8 total, 8 up, 8 in 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:47:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: osdmap e402: 8 total, 8 up, 8 in 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:47:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: osdmap e402: 8 total, 8 up, 8 in 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:47:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:47:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:15 smithi131 ceph-mon[29369]: osdmap e403: 8 total, 8 up, 8 in 2024-09-10T18:47:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:15 smithi131 ceph-mon[29369]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:47:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:47:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:15 smithi131 ceph-mon[40885]: osdmap e403: 8 total, 8 up, 8 in 2024-09-10T18:47:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:15 smithi131 ceph-mon[40885]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:47:15.328 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:47:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:47:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:15 smithi175 ceph-mon[33399]: osdmap e403: 8 total, 8 up, 8 in 2024-09-10T18:47:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:15 smithi175 ceph-mon[33399]: pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:47:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:16 smithi131 ceph-mon[29369]: osdmap e404: 8 total, 8 up, 8 in 2024-09-10T18:47:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:16 smithi131 ceph-mon[40885]: osdmap e404: 8 total, 8 up, 8 in 2024-09-10T18:47:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:16 smithi175 ceph-mon[33399]: osdmap e404: 8 total, 8 up, 8 in 2024-09-10T18:47:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:17 smithi131 ceph-mon[40885]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 717 B/s wr, 2 op/s 2024-09-10T18:47:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:17 smithi131 ceph-mon[29369]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 717 B/s wr, 2 op/s 2024-09-10T18:47:17.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:17 smithi175 ceph-mon[33399]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 717 B/s wr, 2 op/s 2024-09-10T18:47:18.326 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:19 smithi131 ceph-mon[29369]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-09-10T18:47:19.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:19 smithi131 ceph-mon[40885]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-09-10T18:47:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:19 smithi175 ceph-mon[33399]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-09-10T18:47:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:21 smithi175 ceph-mon[33399]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-10T18:47:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:21 smithi131 ceph-mon[29369]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-10T18:47:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:21 smithi131 ceph-mon[40885]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-09-10T18:47:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:22 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:22 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:22 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:23.659 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:23 smithi175 ceph-mon[33399]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 742 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T18:47:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:23 smithi131 ceph-mon[29369]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 742 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T18:47:23.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:23 smithi131 ceph-mon[40885]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 742 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T18:47:23.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:47:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:47:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:47:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:23] "GET /metrics HTTP/1.1" 200 228492 "" "Prometheus/2.43.0" 2024-09-10T18:47:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:24.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:25.404 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:47:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:25.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:25 smithi175 ceph-mon[33399]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-10T18:47:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:25 smithi131 ceph-mon[29369]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-10T18:47:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:25 smithi131 ceph-mon[40885]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-09-10T18:47:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:27 smithi175 ceph-mon[33399]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 999 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:47:27.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:27 smithi131 ceph-mon[29369]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 999 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:47:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:27 smithi131 ceph-mon[40885]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 999 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:47:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:29 smithi175 ceph-mon[33399]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-10T18:47:29.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:29 smithi131 ceph-mon[29369]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-10T18:47:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:29 smithi131 ceph-mon[40885]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-09-10T18:47:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:31 smithi175 ceph-mon[33399]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:31 smithi131 ceph-mon[40885]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:31.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:31 smithi131 ceph-mon[29369]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:47:32.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:47:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:47:33.671 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:33 smithi175 ceph-mon[33399]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:33.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:33 smithi131 ceph-mon[29369]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:33.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:33 smithi131 ceph-mon[40885]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:33.930 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:47:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:47:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:47:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:33] "GET /metrics HTTP/1.1" 200 228466 "" "Prometheus/2.43.0" 2024-09-10T18:47:34.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:34.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:34.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:35.425 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:47:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:35.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:35 smithi175 ceph-mon[33399]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:35.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:35.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:35.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-71"}]: dispatch 2024-09-10T18:47:35.678 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:35 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-71"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[29369]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-71"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-71"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[40885]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-71"}]: dispatch 2024-09-10T18:47:35.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:35 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-71"}]: dispatch 2024-09-10T18:47:36.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:36 smithi131 ceph-mon[40885]: osdmap e405: 8 total, 8 up, 8 in 2024-09-10T18:47:36.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:36 smithi131 ceph-mon[29369]: osdmap e405: 8 total, 8 up, 8 in 2024-09-10T18:47:36.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:36 smithi175 ceph-mon[33399]: osdmap e405: 8 total, 8 up, 8 in 2024-09-10T18:47:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[40885]: pgmap v604: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-10T18:47:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[40885]: osdmap e406: 8 total, 8 up, 8 in 2024-09-10T18:47:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:37.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[29369]: pgmap v604: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-10T18:47:37.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[29369]: osdmap e406: 8 total, 8 up, 8 in 2024-09-10T18:47:37.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:37.829 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:37 smithi175 ceph-mon[33399]: pgmap v604: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-09-10T18:47:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:37 smithi175 ceph-mon[33399]: osdmap e406: 8 total, 8 up, 8 in 2024-09-10T18:47:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:38 smithi131 ceph-mon[29369]: osdmap e407: 8 total, 8 up, 8 in 2024-09-10T18:47:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:38 smithi131 ceph-mon[40885]: osdmap e407: 8 total, 8 up, 8 in 2024-09-10T18:47:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:38 smithi175 ceph-mon[33399]: osdmap e407: 8 total, 8 up, 8 in 2024-09-10T18:47:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:39 smithi131 ceph-mon[29369]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:39 smithi131 ceph-mon[29369]: osdmap e408: 8 total, 8 up, 8 in 2024-09-10T18:47:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:39 smithi131 ceph-mon[40885]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:39 smithi131 ceph-mon[40885]: osdmap e408: 8 total, 8 up, 8 in 2024-09-10T18:47:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:39 smithi175 ceph-mon[33399]: pgmap v607: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:39 smithi175 ceph-mon[33399]: osdmap e408: 8 total, 8 up, 8 in 2024-09-10T18:47:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:40 smithi131 ceph-mon[29369]: osdmap e409: 8 total, 8 up, 8 in 2024-09-10T18:47:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:40 smithi131 ceph-mon[40885]: osdmap e409: 8 total, 8 up, 8 in 2024-09-10T18:47:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:40 smithi175 ceph-mon[33399]: osdmap e409: 8 total, 8 up, 8 in 2024-09-10T18:47:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:41 smithi131 ceph-mon[29369]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:47:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:41 smithi131 ceph-mon[40885]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:47:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:41 smithi175 ceph-mon[33399]: pgmap v610: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:47:43.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:43 smithi131 ceph-mon[29369]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 867 B/s rd, 867 B/s wr, 1 op/s 2024-09-10T18:47:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:43 smithi131 ceph-mon[40885]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 867 B/s rd, 867 B/s wr, 1 op/s 2024-09-10T18:47:43.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:43 smithi175 ceph-mon[33399]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 867 B/s rd, 867 B/s wr, 1 op/s 2024-09-10T18:47:43.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:47:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:47:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:47:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:43] "GET /metrics HTTP/1.1" 200 228466 "" "Prometheus/2.43.0" 2024-09-10T18:47:44.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:44.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:47:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:45 smithi131 ceph-mon[29369]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-10T18:47:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:45 smithi131 ceph-mon[40885]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-10T18:47:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:45 smithi175 ceph-mon[33399]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-09-10T18:47:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:47 smithi131 ceph-mon[29369]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:47 smithi131 ceph-mon[40885]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:47 smithi175 ceph-mon[33399]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:47:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:47:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:49 smithi131 ceph-mon[29369]: pgmap v614: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 207 B/s wr, 2 op/s 2024-09-10T18:47:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:49 smithi131 ceph-mon[40885]: pgmap v614: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 207 B/s wr, 2 op/s 2024-09-10T18:47:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:49 smithi175 ceph-mon[33399]: pgmap v614: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 207 B/s wr, 2 op/s 2024-09-10T18:47:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-73"}]: dispatch 2024-09-10T18:47:50.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-73"}]: dispatch 2024-09-10T18:47:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-73"}]: dispatch 2024-09-10T18:47:50.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-73"}]: dispatch 2024-09-10T18:47:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-73"}]: dispatch 2024-09-10T18:47:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-73"}]: dispatch 2024-09-10T18:47:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:51 smithi131 ceph-mon[29369]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 188 B/s wr, 2 op/s 2024-09-10T18:47:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:51 smithi131 ceph-mon[29369]: osdmap e410: 8 total, 8 up, 8 in 2024-09-10T18:47:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:51 smithi131 ceph-mon[40885]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 188 B/s wr, 2 op/s 2024-09-10T18:47:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:51 smithi131 ceph-mon[40885]: osdmap e410: 8 total, 8 up, 8 in 2024-09-10T18:47:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:51 smithi175 ceph-mon[33399]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 188 B/s wr, 2 op/s 2024-09-10T18:47:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:51 smithi175 ceph-mon[33399]: osdmap e410: 8 total, 8 up, 8 in 2024-09-10T18:47:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[29369]: osdmap e411: 8 total, 8 up, 8 in 2024-09-10T18:47:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[29369]: osdmap e412: 8 total, 8 up, 8 in 2024-09-10T18:47:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[40885]: osdmap e411: 8 total, 8 up, 8 in 2024-09-10T18:47:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[40885]: osdmap e412: 8 total, 8 up, 8 in 2024-09-10T18:47:52.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:52 smithi175 ceph-mon[33399]: osdmap e411: 8 total, 8 up, 8 in 2024-09-10T18:47:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:52 smithi175 ceph-mon[33399]: osdmap e412: 8 total, 8 up, 8 in 2024-09-10T18:47:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:53.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-mon[29369]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:53.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:53.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-mon[29369]: osdmap e413: 8 total, 8 up, 8 in 2024-09-10T18:47:53.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-mon[40885]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:53.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:53.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-mon[40885]: osdmap e413: 8 total, 8 up, 8 in 2024-09-10T18:47:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:53 smithi175 ceph-mon[33399]: pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:53 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:47:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:53 smithi175 ceph-mon[33399]: osdmap e413: 8 total, 8 up, 8 in 2024-09-10T18:47:53.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:47:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:47:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:47:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:53] "GET /metrics HTTP/1.1" 200 228490 "" "Prometheus/2.43.0" 2024-09-10T18:47:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:54.827 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:47:54 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:47:54.555+0000 7f6e75ee9640 -1 snap_mapper.add_oid found existing snaps mapped on 64:7504564d:test-rados-api-smithi131-80444-76::foo:28, removing 2024-09-10T18:47:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:47:54.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:47:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:47:54.555+0000 7f38468bd640 -1 snap_mapper.add_oid found existing snaps mapped on 64:7504564d:test-rados-api-smithi131-80444-76::foo:28, removing 2024-09-10T18:47:54.927 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:47:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:47:54.555+0000 7fa226ee1640 -1 snap_mapper.add_oid found existing snaps mapped on 64:7504564d:test-rados-api-smithi131-80444-76::foo:28, removing 2024-09-10T18:47:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:47:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[29369]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[29369]: osdmap e414: 8 total, 8 up, 8 in 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-75"}]: dispatch 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-75"}]: dispatch 2024-09-10T18:47:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[40885]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[40885]: osdmap e414: 8 total, 8 up, 8 in 2024-09-10T18:47:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-75"}]: dispatch 2024-09-10T18:47:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-75"}]: dispatch 2024-09-10T18:47:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-mon[33399]: pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:47:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-mon[33399]: osdmap e414: 8 total, 8 up, 8 in 2024-09-10T18:47:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-75"}]: dispatch 2024-09-10T18:47:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-75"}]: dispatch 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[29369]: osdmap e415: 8 total, 8 up, 8 in 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[29369]: pgmap v624: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[29369]: osdmap e416: 8 total, 8 up, 8 in 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[40885]: osdmap e415: 8 total, 8 up, 8 in 2024-09-10T18:47:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[40885]: pgmap v624: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:57.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[40885]: osdmap e416: 8 total, 8 up, 8 in 2024-09-10T18:47:57.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:57.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:56 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:56 smithi175 ceph-mon[33399]: osdmap e415: 8 total, 8 up, 8 in 2024-09-10T18:47:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:56 smithi175 ceph-mon[33399]: pgmap v624: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:47:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:56 smithi175 ceph-mon[33399]: osdmap e416: 8 total, 8 up, 8 in 2024-09-10T18:47:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:56 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:47:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-mon[29369]: osdmap e417: 8 total, 8 up, 8 in 2024-09-10T18:47:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:58.827 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:47:58.560+0000 7fbb1ea9d640 -1 snap_mapper.add_oid found existing snaps mapped on 64:7ea36443:test-rados-api-smithi131-80444-78::foo:29, removing 2024-09-10T18:47:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-mon[40885]: osdmap e417: 8 total, 8 up, 8 in 2024-09-10T18:47:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:58.828 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:47:58 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0[45274]: 2024-09-10T18:47:58.560+0000 7f169ef83640 -1 snap_mapper.add_oid found existing snaps mapped on 64:7ea36443:test-rados-api-smithi131-80444-78::foo:29, removing 2024-09-10T18:47:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:47:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:58 smithi175 ceph-mon[33399]: osdmap e417: 8 total, 8 up, 8 in 2024-09-10T18:47:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:47:58.927 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:47:58 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:47:58.560+0000 7fb37991f640 -1 snap_mapper.add_oid found existing snaps mapped on 64:7ea36443:test-rados-api-smithi131-80444-78::foo:29, removing 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[29369]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[29369]: osdmap e418: 8 total, 8 up, 8 in 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-77"}]: dispatch 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-77"}]: dispatch 2024-09-10T18:47:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[40885]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-10T18:47:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[40885]: osdmap e418: 8 total, 8 up, 8 in 2024-09-10T18:47:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-77"}]: dispatch 2024-09-10T18:47:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:47:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-77"}]: dispatch 2024-09-10T18:47:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:59 smithi175 ceph-mon[33399]: pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 4 op/s 2024-09-10T18:47:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:59 smithi175 ceph-mon[33399]: osdmap e418: 8 total, 8 up, 8 in 2024-09-10T18:47:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:47:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-77"}]: dispatch 2024-09-10T18:47:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:47:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-77"}]: dispatch 2024-09-10T18:48:00.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:00 smithi131 ceph-mon[29369]: osdmap e419: 8 total, 8 up, 8 in 2024-09-10T18:48:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:00 smithi131 ceph-mon[40885]: osdmap e419: 8 total, 8 up, 8 in 2024-09-10T18:48:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:00 smithi175 ceph-mon[33399]: osdmap e419: 8 total, 8 up, 8 in 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[29369]: pgmap v630: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[29369]: osdmap e420: 8 total, 8 up, 8 in 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[40885]: pgmap v630: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[40885]: osdmap e420: 8 total, 8 up, 8 in 2024-09-10T18:48:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:01.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:01 smithi175 ceph-mon[33399]: pgmap v630: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 4 op/s 2024-09-10T18:48:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:01 smithi175 ceph-mon[33399]: osdmap e420: 8 total, 8 up, 8 in 2024-09-10T18:48:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[29369]: osdmap e421: 8 total, 8 up, 8 in 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:02.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[40885]: osdmap e421: 8 total, 8 up, 8 in 2024-09-10T18:48:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:02 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:02 smithi175 ceph-mon[33399]: osdmap e421: 8 total, 8 up, 8 in 2024-09-10T18:48:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:02 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:03 smithi175 ceph-mon[33399]: pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:03 smithi175 ceph-mon[33399]: osdmap e422: 8 total, 8 up, 8 in 2024-09-10T18:48:03.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:48:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:48:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:48:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:03] "GET /metrics HTTP/1.1" 200 228496 "" "Prometheus/2.43.0" 2024-09-10T18:48:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:03 smithi131 ceph-mon[29369]: pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:03 smithi131 ceph-mon[29369]: osdmap e422: 8 total, 8 up, 8 in 2024-09-10T18:48:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:03 smithi131 ceph-mon[40885]: pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:03 smithi131 ceph-mon[40885]: osdmap e422: 8 total, 8 up, 8 in 2024-09-10T18:48:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:04 smithi175 ceph-mon[33399]: osdmap e423: 8 total, 8 up, 8 in 2024-09-10T18:48:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.3"}]: dispatch 2024-09-10T18:48:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:04 smithi175 ceph-mon[33399]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.3"}]: dispatch 2024-09-10T18:48:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:04 smithi175 ceph-mon[33399]: 223.3 deep-scrub starts 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[40885]: osdmap e423: 8 total, 8 up, 8 in 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.3"}]: dispatch 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[40885]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.3"}]: dispatch 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[40885]: 223.3 deep-scrub starts 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[29369]: osdmap e423: 8 total, 8 up, 8 in 2024-09-10T18:48:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.3"}]: dispatch 2024-09-10T18:48:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[29369]: from='mon.? v1:172.21.15.175:0/4208463607' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.3"}]: dispatch 2024-09-10T18:48:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:05.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:04 smithi131 ceph-mon[29369]: 223.3 deep-scrub starts 2024-09-10T18:48:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:48:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:05 smithi175 ceph-mon[33399]: 223.3 deep-scrub ok 2024-09-10T18:48:05.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:05 smithi175 ceph-mon[33399]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:48:06.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:05 smithi131 ceph-mon[29369]: 223.3 deep-scrub ok 2024-09-10T18:48:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:05 smithi131 ceph-mon[29369]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:48:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:05 smithi131 ceph-mon[40885]: 223.3 deep-scrub ok 2024-09-10T18:48:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:05 smithi131 ceph-mon[40885]: pgmap v636: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T18:48:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:07 smithi175 ceph-mon[33399]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-09-10T18:48:08.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:07 smithi131 ceph-mon[29369]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-09-10T18:48:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:07 smithi131 ceph-mon[40885]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.0 KiB/s wr, 2 op/s 2024-09-10T18:48:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:09 smithi175 ceph-mon[33399]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-10T18:48:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:09 smithi131 ceph-mon[29369]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-10T18:48:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:09 smithi131 ceph-mon[40885]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-09-10T18:48:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:11 smithi175 ceph-mon[33399]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-10T18:48:12.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:11 smithi131 ceph-mon[29369]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-10T18:48:12.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:11 smithi131 ceph-mon[40885]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 3 op/s 2024-09-10T18:48:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:13 smithi175 ceph-mon[33399]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 733 B/s wr, 2 op/s 2024-09-10T18:48:13.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:48:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:48:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:13 smithi131 ceph-mon[29369]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 733 B/s wr, 2 op/s 2024-09-10T18:48:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:48:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:13] "GET /metrics HTTP/1.1" 200 228496 "" "Prometheus/2.43.0" 2024-09-10T18:48:14.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:13 smithi131 ceph-mon[40885]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 733 B/s wr, 2 op/s 2024-09-10T18:48:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:15.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:48:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:15 smithi175 ceph-mon[33399]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 950 B/s rd, 95 B/s wr, 1 op/s 2024-09-10T18:48:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:15 smithi131 ceph-mon[29369]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 950 B/s rd, 95 B/s wr, 1 op/s 2024-09-10T18:48:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:15 smithi131 ceph-mon[40885]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 950 B/s rd, 95 B/s wr, 1 op/s 2024-09-10T18:48:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:17 smithi175 ceph-mon[33399]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:48:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:18.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:17 smithi131 ceph-mon[29369]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:48:18.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:18.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:17 smithi131 ceph-mon[40885]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:48:18.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:19 smithi175 ceph-mon[33399]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:48:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:19 smithi131 ceph-mon[29369]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:48:20.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:19 smithi131 ceph-mon[40885]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:48:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:21 smithi175 ceph-mon[33399]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:48:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:21 smithi131 ceph-mon[29369]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:48:22.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:21 smithi131 ceph-mon[40885]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:48:23.927 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:48:23.609+0000 7fb37b122640 -1 snap_mapper.add_oid found existing snaps mapped on 223:c3a6bae0:test-rados-api-smithi131-80444-80::foo:2, removing 2024-09-10T18:48:23.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:48:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-mon[33399]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:48:23.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:23.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:23.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-79"}]: dispatch 2024-09-10T18:48:23.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:23 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-79"}]: dispatch 2024-09-10T18:48:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:23] "GET /metrics HTTP/1.1" 200 228486 "" "Prometheus/2.43.0" 2024-09-10T18:48:24.077 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:48:23.609+0000 7f6e7b6f4640 -1 snap_mapper.add_oid found existing snaps mapped on 223:c3a6bae0:test-rados-api-smithi131-80444-80::foo:2, removing 2024-09-10T18:48:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[29369]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:48:24.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:24.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:24.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-79"}]: dispatch 2024-09-10T18:48:24.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-79"}]: dispatch 2024-09-10T18:48:24.078 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1[51510]: 2024-09-10T18:48:23.609+0000 7fd35fa33640 -1 snap_mapper.add_oid found existing snaps mapped on 223:c3a6bae0:test-rados-api-smithi131-80444-80::foo:2, removing 2024-09-10T18:48:24.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[40885]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:48:24.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:24.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:24.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-79"}]: dispatch 2024-09-10T18:48:24.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:23 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-79"}]: dispatch 2024-09-10T18:48:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:24 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:24 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:25.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:24 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:48:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:25 smithi175 ceph-mon[33399]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:25 smithi175 ceph-mon[33399]: osdmap e424: 8 total, 8 up, 8 in 2024-09-10T18:48:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:25 smithi131 ceph-mon[29369]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:25 smithi131 ceph-mon[29369]: osdmap e424: 8 total, 8 up, 8 in 2024-09-10T18:48:26.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:25 smithi131 ceph-mon[40885]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:26.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:25 smithi131 ceph-mon[40885]: osdmap e424: 8 total, 8 up, 8 in 2024-09-10T18:48:26.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:26 smithi175 ceph-mon[33399]: osdmap e425: 8 total, 8 up, 8 in 2024-09-10T18:48:26.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:26.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:26 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:27.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:26 smithi131 ceph-mon[29369]: osdmap e425: 8 total, 8 up, 8 in 2024-09-10T18:48:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:27.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:26 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:27.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:26 smithi131 ceph-mon[40885]: osdmap e425: 8 total, 8 up, 8 in 2024-09-10T18:48:27.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:27.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:26 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: pgmap v649: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: osdmap e426: 8 total, 8 up, 8 in 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:27 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: pgmap v649: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: osdmap e426: 8 total, 8 up, 8 in 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:28.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: pgmap v649: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: osdmap e426: 8 total, 8 up, 8 in 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:28.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:27 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:28 smithi175 ceph-mon[33399]: osdmap e427: 8 total, 8 up, 8 in 2024-09-10T18:48:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:28 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:28.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:28 smithi175 ceph-mon[33399]: osdmap e428: 8 total, 8 up, 8 in 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[29369]: osdmap e427: 8 total, 8 up, 8 in 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[29369]: osdmap e428: 8 total, 8 up, 8 in 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:29.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[40885]: osdmap e427: 8 total, 8 up, 8 in 2024-09-10T18:48:29.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:29.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:29.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:29.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:28 smithi131 ceph-mon[40885]: osdmap e428: 8 total, 8 up, 8 in 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: osdmap e429: 8 total, 8 up, 8 in 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:29 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: osdmap e429: 8 total, 8 up, 8 in 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:29.978 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: pgmap v652: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-09-10T18:48:29.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:29.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:29.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:29.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: osdmap e429: 8 total, 8 up, 8 in 2024-09-10T18:48:29.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:29.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:29 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:31 smithi175 ceph-mon[33399]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:48:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:31 smithi175 ceph-mon[33399]: osdmap e430: 8 total, 8 up, 8 in 2024-09-10T18:48:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:48:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:31 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:48:32.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[40885]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:48:32.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:32.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[40885]: osdmap e430: 8 total, 8 up, 8 in 2024-09-10T18:48:32.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:48:32.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:48:32.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[29369]: pgmap v655: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:48:32.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:32.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[29369]: osdmap e430: 8 total, 8 up, 8 in 2024-09-10T18:48:32.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:48:32.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:31 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T18:48:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T18:48:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: osdmap e431: 8 total, 8 up, 8 in 2024-09-10T18:48:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:48:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:48:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:32.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:32.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: osdmap e431: 8 total, 8 up, 8 in 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T18:48:33.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: osdmap e431: 8 total, 8 up, 8 in 2024-09-10T18:48:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:48:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T18:48:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:33.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:48:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T18:48:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: osdmap e432: 8 total, 8 up, 8 in 2024-09-10T18:48:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:33.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:33.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:33.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:33] "GET /metrics HTTP/1.1" 200 228467 "" "Prometheus/2.43.0" 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: osdmap e432: 8 total, 8 up, 8 in 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:34.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: pgmap v658: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: osdmap e432: 8 total, 8 up, 8 in 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:48:34.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:48:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: osdmap e433: 8 total, 8 up, 8 in 2024-09-10T18:48:35.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:35.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:35.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-81"}]: dispatch 2024-09-10T18:48:35.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-81"}]: dispatch 2024-09-10T18:48:35.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:35.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:35.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: osdmap e433: 8 total, 8 up, 8 in 2024-09-10T18:48:35.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:35.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:35.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-81"}]: dispatch 2024-09-10T18:48:35.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-81"}]: dispatch 2024-09-10T18:48:35.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:34 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:35.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:35.080 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: osdmap e433: 8 total, 8 up, 8 in 2024-09-10T18:48:35.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:35.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:35.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-81"}]: dispatch 2024-09-10T18:48:35.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-81"}]: dispatch 2024-09-10T18:48:35.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:34 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:48:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:35 smithi131 ceph-mon[29369]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 11 KiB/s wr, 49 op/s 2024-09-10T18:48:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:35 smithi131 ceph-mon[29369]: osdmap e434: 8 total, 8 up, 8 in 2024-09-10T18:48:36.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:35 smithi131 ceph-mon[40885]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 11 KiB/s wr, 49 op/s 2024-09-10T18:48:36.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:35 smithi131 ceph-mon[40885]: osdmap e434: 8 total, 8 up, 8 in 2024-09-10T18:48:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:35 smithi175 ceph-mon[33399]: pgmap v661: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 11 KiB/s wr, 49 op/s 2024-09-10T18:48:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:35 smithi175 ceph-mon[33399]: osdmap e434: 8 total, 8 up, 8 in 2024-09-10T18:48:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:36 smithi131 ceph-mon[29369]: osdmap e435: 8 total, 8 up, 8 in 2024-09-10T18:48:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:36 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:37.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:36 smithi131 ceph-mon[40885]: osdmap e435: 8 total, 8 up, 8 in 2024-09-10T18:48:37.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:37.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:36 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:36 smithi175 ceph-mon[33399]: osdmap e435: 8 total, 8 up, 8 in 2024-09-10T18:48:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:37.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:36 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: pgmap v664: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 9.0 KiB/s wr, 48 op/s 2024-09-10T18:48:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: osdmap e436: 8 total, 8 up, 8 in 2024-09-10T18:48:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: pgmap v664: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 9.0 KiB/s wr, 48 op/s 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: osdmap e436: 8 total, 8 up, 8 in 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:38.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:37 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: pgmap v664: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 19 KiB/s rd, 9.0 KiB/s wr, 48 op/s 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: osdmap e436: 8 total, 8 up, 8 in 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:37 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[29369]: osdmap e437: 8 total, 8 up, 8 in 2024-09-10T18:48:39.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:39.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[40885]: osdmap e437: 8 total, 8 up, 8 in 2024-09-10T18:48:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:39.079 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:38 smithi175 ceph-mon[33399]: osdmap e437: 8 total, 8 up, 8 in 2024-09-10T18:48:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 99 KiB/s rd, 767 B/s wr, 169 op/s 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: osdmap e438: 8 total, 8 up, 8 in 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: osdmap e439: 8 total, 8 up, 8 in 2024-09-10T18:48:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 99 KiB/s rd, 767 B/s wr, 169 op/s 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: osdmap e438: 8 total, 8 up, 8 in 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: osdmap e439: 8 total, 8 up, 8 in 2024-09-10T18:48:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: pgmap v667: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 99 KiB/s rd, 767 B/s wr, 169 op/s 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: osdmap e438: 8 total, 8 up, 8 in 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: osdmap e439: 8 total, 8 up, 8 in 2024-09-10T18:48:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:41.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:40 smithi131 ceph-mon[29369]: osdmap e440: 8 total, 8 up, 8 in 2024-09-10T18:48:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:41.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:40 smithi131 ceph-mon[40885]: osdmap e440: 8 total, 8 up, 8 in 2024-09-10T18:48:41.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:41.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:41.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:40 smithi175 ceph-mon[33399]: osdmap e440: 8 total, 8 up, 8 in 2024-09-10T18:48:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:41 smithi131 ceph-mon[29369]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 99 KiB/s rd, 3.2 KiB/s wr, 170 op/s 2024-09-10T18:48:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:41 smithi131 ceph-mon[40885]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 99 KiB/s rd, 3.2 KiB/s wr, 170 op/s 2024-09-10T18:48:42.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:41 smithi175 ceph-mon[33399]: pgmap v670: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 99 KiB/s rd, 3.2 KiB/s wr, 170 op/s 2024-09-10T18:48:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:42 smithi131 ceph-mon[29369]: osdmap e441: 8 total, 8 up, 8 in 2024-09-10T18:48:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:42 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:42 smithi131 ceph-mon[29369]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:42 smithi131 ceph-mon[40885]: osdmap e441: 8 total, 8 up, 8 in 2024-09-10T18:48:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:42 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:42 smithi131 ceph-mon[40885]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:42 smithi175 ceph-mon[33399]: osdmap e441: 8 total, 8 up, 8 in 2024-09-10T18:48:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:42 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:43.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:42 smithi175 ceph-mon[33399]: pgmap v673: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:43] "GET /metrics HTTP/1.1" 200 228467 "" "Prometheus/2.43.0" 2024-09-10T18:48:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[29369]: osdmap e442: 8 total, 8 up, 8 in 2024-09-10T18:48:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-83"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-83"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[40885]: osdmap e442: 8 total, 8 up, 8 in 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-83"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-83"}]: dispatch 2024-09-10T18:48:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:43 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:44.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:48:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-mon[33399]: osdmap e442: 8 total, 8 up, 8 in 2024-09-10T18:48:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-83"}]: dispatch 2024-09-10T18:48:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-83"}]: dispatch 2024-09-10T18:48:44.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:43 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:44 smithi131 ceph-mon[29369]: osdmap e443: 8 total, 8 up, 8 in 2024-09-10T18:48:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:44 smithi131 ceph-mon[29369]: pgmap v676: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:48:45.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:44 smithi131 ceph-mon[40885]: osdmap e443: 8 total, 8 up, 8 in 2024-09-10T18:48:45.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:44 smithi131 ceph-mon[40885]: pgmap v676: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:48:45.077 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:44 smithi175 ceph-mon[33399]: osdmap e443: 8 total, 8 up, 8 in 2024-09-10T18:48:45.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:44 smithi175 ceph-mon[33399]: pgmap v676: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:48:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:48:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:45 smithi131 ceph-mon[29369]: osdmap e444: 8 total, 8 up, 8 in 2024-09-10T18:48:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:45 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:45 smithi131 ceph-mon[40885]: osdmap e444: 8 total, 8 up, 8 in 2024-09-10T18:48:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:45 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:45 smithi175 ceph-mon[33399]: osdmap e444: 8 total, 8 up, 8 in 2024-09-10T18:48:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:45 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[29369]: osdmap e445: 8 total, 8 up, 8 in 2024-09-10T18:48:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[29369]: pgmap v679: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:48:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[40885]: osdmap e445: 8 total, 8 up, 8 in 2024-09-10T18:48:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:47.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:46 smithi131 ceph-mon[40885]: pgmap v679: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:48:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:46 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:46 smithi175 ceph-mon[33399]: osdmap e445: 8 total, 8 up, 8 in 2024-09-10T18:48:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:46 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:46 smithi175 ceph-mon[33399]: pgmap v679: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T18:48:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[29369]: osdmap e446: 8 total, 8 up, 8 in 2024-09-10T18:48:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[40885]: osdmap e446: 8 total, 8 up, 8 in 2024-09-10T18:48:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:47 smithi175 ceph-mon[33399]: osdmap e446: 8 total, 8 up, 8 in 2024-09-10T18:48:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:47 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:48:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[29369]: osdmap e447: 8 total, 8 up, 8 in 2024-09-10T18:48:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[29369]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[40885]: osdmap e447: 8 total, 8 up, 8 in 2024-09-10T18:48:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:48 smithi131 ceph-mon[40885]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_tier","val": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:48:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:48 smithi175 ceph-mon[33399]: osdmap e447: 8 total, 8 up, 8 in 2024-09-10T18:48:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:48 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:48 smithi175 ceph-mon[33399]: pgmap v682: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[40885]: osdmap e448: 8 total, 8 up, 8 in 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[29369]: osdmap e448: 8 total, 8 up, 8 in 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:49 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:49 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:49 smithi175 ceph-mon[33399]: osdmap e448: 8 total, 8 up, 8 in 2024-09-10T18:48:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:49 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:50 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:50 smithi131 ceph-mon[40885]: osdmap e449: 8 total, 8 up, 8 in 2024-09-10T18:48:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:50 smithi131 ceph-mon[40885]: pgmap v685: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:50 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:50 smithi131 ceph-mon[29369]: osdmap e449: 8 total, 8 up, 8 in 2024-09-10T18:48:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:50 smithi131 ceph-mon[29369]: pgmap v685: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:51.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:50 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:48:51.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:50 smithi175 ceph-mon[33399]: osdmap e449: 8 total, 8 up, 8 in 2024-09-10T18:48:51.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:50 smithi175 ceph-mon[33399]: pgmap v685: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:51 smithi131 ceph-mon[29369]: osdmap e450: 8 total, 8 up, 8 in 2024-09-10T18:48:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:51 smithi131 ceph-mon[40885]: osdmap e450: 8 total, 8 up, 8 in 2024-09-10T18:48:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:51 smithi175 ceph-mon[33399]: osdmap e450: 8 total, 8 up, 8 in 2024-09-10T18:48:52.837 INFO:tasks.workunit.client.0.smithi131.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18113 ms) 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22117 ms) 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3005 ms) 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3015 ms) 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3021 ms) 2024-09-10T18:48:52.838 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3003 ms) 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4017 ms) 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38884 ms) 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16598 ms) 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-10T18:48:52.839 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4036 ms) 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3027 ms) 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24391 ms) 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15068 ms) 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-10T18:48:52.840 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5028 ms) 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4013 ms) 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: waiting for scrubs... 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: done waiting 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25093 ms) 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.841 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10054 ms) 2024-09-10T18:48:52.842 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-10T18:48:52.842 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.842 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9073 ms) 2024-09-10T18:48:52.842 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-10T18:48:52.842 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:48:52.842 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9075 ms) 2024-09-10T18:48:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:52 smithi175 ceph-mon[33399]: osdmap e451: 8 total, 8 up, 8 in 2024-09-10T18:48:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-85"}]: dispatch 2024-09-10T18:48:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-85"}]: dispatch 2024-09-10T18:48:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:52 smithi175 ceph-mon[33399]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[40885]: osdmap e451: 8 total, 8 up, 8 in 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-85"}]: dispatch 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-85"}]: dispatch 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[40885]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:53.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[29369]: osdmap e451: 8 total, 8 up, 8 in 2024-09-10T18:48:53.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:53.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:48:53.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-85"}]: dispatch 2024-09-10T18:48:53.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-85"}]: dispatch 2024-09-10T18:48:53.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:52 smithi131 ceph-mon[29369]: pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:48:54.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:53 smithi131 ceph-mon[29369]: osdmap e452: 8 total, 8 up, 8 in 2024-09-10T18:48:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:53 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:48:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:53] "GET /metrics HTTP/1.1" 200 228555 "" "Prometheus/2.43.0" 2024-09-10T18:48:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:53 smithi131 ceph-mon[40885]: osdmap e452: 8 total, 8 up, 8 in 2024-09-10T18:48:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:53 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:53 smithi175 ceph-mon[33399]: osdmap e452: 8 total, 8 up, 8 in 2024-09-10T18:48:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:53 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:48:54.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:48:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:48:55.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:48:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:48:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:48:55.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:54 smithi175 ceph-mon[33399]: osdmap e453: 8 total, 8 up, 8 in 2024-09-10T18:48:55.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:55.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:54 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:55.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:54 smithi175 ceph-mon[33399]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[40885]: osdmap e453: 8 total, 8 up, 8 in 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[40885]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[29369]: osdmap e453: 8 total, 8 up, 8 in 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:48:55.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:54 smithi131 ceph-mon[29369]: pgmap v691: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:48:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:55 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:55 smithi175 ceph-mon[33399]: osdmap e454: 8 total, 8 up, 8 in 2024-09-10T18:48:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[29369]: osdmap e454: 8 total, 8 up, 8 in 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:48:56.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[40885]: osdmap e454: 8 total, 8 up, 8 in 2024-09-10T18:48:56.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:48:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:56 smithi175 ceph-mon[33399]: osdmap e455: 8 total, 8 up, 8 in 2024-09-10T18:48:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:56 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:56 smithi175 ceph-mon[33399]: pgmap v694: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[29369]: osdmap e455: 8 total, 8 up, 8 in 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[29369]: pgmap v694: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[40885]: osdmap e455: 8 total, 8 up, 8 in 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:57.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:48:57.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:56 smithi131 ceph-mon[40885]: pgmap v694: 260 pgs: 64 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T18:48:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: osdmap e456: 8 total, 8 up, 8 in 2024-09-10T18:48:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]: dispatch 2024-09-10T18:48:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]: dispatch 2024-09-10T18:48:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]': finished 2024-09-10T18:48:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: osdmap e457: 8 total, 8 up, 8 in 2024-09-10T18:48:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:58.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: osdmap e456: 8 total, 8 up, 8 in 2024-09-10T18:48:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]: dispatch 2024-09-10T18:48:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]: dispatch 2024-09-10T18:48:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]': finished 2024-09-10T18:48:58.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: osdmap e457: 8 total, 8 up, 8 in 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: osdmap e456: 8 total, 8 up, 8 in 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]: dispatch 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]: dispatch 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_tier","val": "test-rados-api-smithi131-80444-89-test-flush"}]': finished 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: osdmap e457: 8 total, 8 up, 8 in 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:58.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:48:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:58 smithi175 ceph-mon[33399]: pgmap v697: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:58 smithi175 ceph-mon[33399]: osdmap e458: 8 total, 8 up, 8 in 2024-09-10T18:48:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:48:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:59.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[29369]: pgmap v697: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:59.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:59.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[29369]: osdmap e458: 8 total, 8 up, 8 in 2024-09-10T18:48:59.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:59.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:59.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[40885]: pgmap v697: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:48:59.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:48:59.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[40885]: osdmap e458: 8 total, 8 up, 8 in 2024-09-10T18:48:59.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:48:59.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:48:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:49:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:49:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:00 smithi175 ceph-mon[33399]: osdmap e459: 8 total, 8 up, 8 in 2024-09-10T18:49:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:00 smithi175 ceph-mon[33399]: pgmap v700: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:49:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:00 smithi131 ceph-mon[29369]: osdmap e459: 8 total, 8 up, 8 in 2024-09-10T18:49:01.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:00 smithi131 ceph-mon[29369]: pgmap v700: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:01.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:00 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:49:01.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:00 smithi131 ceph-mon[40885]: osdmap e459: 8 total, 8 up, 8 in 2024-09-10T18:49:01.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:00 smithi131 ceph-mon[40885]: pgmap v700: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:49:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:01 smithi175 ceph-mon[33399]: osdmap e460: 8 total, 8 up, 8 in 2024-09-10T18:49:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-87"}]: dispatch 2024-09-10T18:49:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-87"}]: dispatch 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[29369]: osdmap e460: 8 total, 8 up, 8 in 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-87"}]: dispatch 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-87"}]: dispatch 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[40885]: osdmap e460: 8 total, 8 up, 8 in 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-87"}]: dispatch 2024-09-10T18:49:02.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-87"}]: dispatch 2024-09-10T18:49:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:02 smithi175 ceph-mon[33399]: osdmap e461: 8 total, 8 up, 8 in 2024-09-10T18:49:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:02 smithi175 ceph-mon[33399]: pgmap v703: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:02 smithi131 ceph-mon[29369]: osdmap e461: 8 total, 8 up, 8 in 2024-09-10T18:49:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:02 smithi131 ceph-mon[29369]: pgmap v703: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:02 smithi131 ceph-mon[40885]: osdmap e461: 8 total, 8 up, 8 in 2024-09-10T18:49:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:02 smithi131 ceph-mon[40885]: pgmap v703: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:03] "GET /metrics HTTP/1.1" 200 226116 "" "Prometheus/2.43.0" 2024-09-10T18:49:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[29369]: osdmap e462: 8 total, 8 up, 8 in 2024-09-10T18:49:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:04.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[40885]: osdmap e462: 8 total, 8 up, 8 in 2024-09-10T18:49:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:03 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:04.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:49:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:49:04.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:03 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:04.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:03 smithi175 ceph-mon[33399]: osdmap e462: 8 total, 8 up, 8 in 2024-09-10T18:49:04.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:04.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:04.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:03 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:05.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:49:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:04 smithi175 ceph-mon[33399]: osdmap e463: 8 total, 8 up, 8 in 2024-09-10T18:49:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:49:05.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:49:05.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:04 smithi175 ceph-mon[33399]: pgmap v706: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[29369]: osdmap e463: 8 total, 8 up, 8 in 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[29369]: pgmap v706: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[40885]: osdmap e463: 8 total, 8 up, 8 in 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:05.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:49:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:49:05.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:04 smithi131 ceph-mon[40885]: pgmap v706: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:49:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:05 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:49:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:05 smithi175 ceph-mon[33399]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T18:49:06.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:05 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:49:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:05 smithi131 ceph-mon[29369]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T18:49:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:05 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:49:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:05 smithi131 ceph-mon[40885]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T18:49:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:06 smithi175 ceph-mon[33399]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T18:49:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:06 smithi175 ceph-mon[33399]: pgmap v709: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:49:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:06 smithi131 ceph-mon[29369]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T18:49:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:06 smithi131 ceph-mon[29369]: pgmap v709: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:49:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:06 smithi131 ceph-mon[40885]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T18:49:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:06 smithi131 ceph-mon[40885]: pgmap v709: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:49:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:08 smithi175 ceph-mon[33399]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T18:49:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-90"}]: dispatch 2024-09-10T18:49:08.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:08 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-90"}]: dispatch 2024-09-10T18:49:08.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[29369]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-90"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-90"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[40885]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-90"}]: dispatch 2024-09-10T18:49:08.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:08 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-90"}]: dispatch 2024-09-10T18:49:09.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:09 smithi175 ceph-mon[33399]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T18:49:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:09 smithi175 ceph-mon[33399]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:49:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:09 smithi175 ceph-mon[33399]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T18:49:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:09.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[29369]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[29369]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[29369]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[40885]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T18:49:09.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[40885]: pgmap v712: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:49:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[40885]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T18:49:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:09.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:10 smithi175 ceph-mon[33399]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T18:49:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:10 smithi175 ceph-mon[33399]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[29369]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[29369]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[40885]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:10 smithi131 ceph-mon[40885]: pgmap v715: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:49:12.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:11 smithi175 ceph-mon[33399]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T18:49:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:11 smithi131 ceph-mon[29369]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T18:49:12.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:11 smithi131 ceph-mon[40885]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T18:49:13.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:12 smithi175 ceph-mon[33399]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T18:49:13.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:13.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:13.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-92"}]: dispatch 2024-09-10T18:49:13.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-92"}]: dispatch 2024-09-10T18:49:13.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:12 smithi175 ceph-mon[33399]: pgmap v718: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[29369]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T18:49:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-92"}]: dispatch 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-92"}]: dispatch 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[29369]: pgmap v718: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[40885]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-92"}]: dispatch 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-92"}]: dispatch 2024-09-10T18:49:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:12 smithi131 ceph-mon[40885]: pgmap v718: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:14.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:13 smithi131 ceph-mon[29369]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T18:49:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:13 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:49:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:13] "GET /metrics HTTP/1.1" 200 226116 "" "Prometheus/2.43.0" 2024-09-10T18:49:14.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:13 smithi131 ceph-mon[40885]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T18:49:14.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:13 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:13 smithi175 ceph-mon[33399]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T18:49:14.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:13 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:14.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:49:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:49:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[29369]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T18:49:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[29369]: pgmap v721: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:49:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[40885]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T18:49:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:14 smithi131 ceph-mon[40885]: pgmap v721: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:49:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:49:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:14 smithi175 ceph-mon[33399]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T18:49:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:14 smithi175 ceph-mon[33399]: pgmap v721: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:49:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[29369]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T18:49:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[40885]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T18:49:16.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:15 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:15 smithi175 ceph-mon[33399]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T18:49:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:16 smithi131 ceph-mon[29369]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T18:49:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:16 smithi131 ceph-mon[29369]: pgmap v724: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:49:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:16 smithi131 ceph-mon[40885]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T18:49:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:16 smithi131 ceph-mon[40885]: pgmap v724: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:49:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:16 smithi175 ceph-mon[33399]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T18:49:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:16 smithi175 ceph-mon[33399]: pgmap v724: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T18:49:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:17 smithi131 ceph-mon[29369]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T18:49:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:17 smithi131 ceph-mon[40885]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T18:49:18.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:17 smithi175 ceph-mon[33399]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T18:49:18.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:19.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:18 smithi131 ceph-mon[29369]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-10T18:49:19.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:18 smithi131 ceph-mon[40885]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-10T18:49:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:18 smithi175 ceph-mon[33399]: pgmap v726: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-09-10T18:49:21.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:21 smithi175 ceph-mon[33399]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T18:49:21.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:21 smithi131 ceph-mon[29369]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T18:49:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:21 smithi131 ceph-mon[40885]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T18:49:22.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:22 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:22.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:22 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:22 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:23.747 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:23 smithi175 ceph-mon[33399]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:49:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:23 smithi131 ceph-mon[29369]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:49:23.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:23 smithi131 ceph-mon[40885]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-10T18:49:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:49:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:23] "GET /metrics HTTP/1.1" 200 228544 "" "Prometheus/2.43.0" 2024-09-10T18:49:24.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:49:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:49:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:49:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:25 smithi131 ceph-mon[29369]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 972 B/s wr, 4 op/s 2024-09-10T18:49:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:25 smithi131 ceph-mon[40885]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 972 B/s wr, 4 op/s 2024-09-10T18:49:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:25 smithi175 ceph-mon[33399]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 972 B/s wr, 4 op/s 2024-09-10T18:49:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:27 smithi131 ceph-mon[29369]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:49:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:27 smithi131 ceph-mon[40885]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:49:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:27 smithi175 ceph-mon[33399]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-09-10T18:49:28.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:28 smithi131 ceph-mon[29369]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T18:49:28.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:28 smithi131 ceph-mon[40885]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T18:49:28.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:28 smithi175 ceph-mon[33399]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T18:49:29.727 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:29 smithi131 ceph-mon[29369]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:29.728 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:29 smithi131 ceph-mon[40885]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:29 smithi175 ceph-mon[33399]: pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:31.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:31 smithi131 ceph-mon[29369]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:31 smithi131 ceph-mon[40885]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:31 smithi175 ceph-mon[33399]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:33 smithi175 ceph-mon[33399]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T18:49:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:33 smithi175 ceph-mon[33399]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:33.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[29369]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[29369]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[40885]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[40885]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:49:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:33] "GET /metrics HTTP/1.1" 200 228521 "" "Prometheus/2.43.0" 2024-09-10T18:49:34.128 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:49:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:49:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi175 to 3248M 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: Adjusting osd_memory_target on smithi131 to 3248M 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:34.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:34 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi175 to 3248M 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: Adjusting osd_memory_target on smithi131 to 3248M 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi175 to 3248M 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: Adjusting osd_memory_target on smithi131 to 3248M 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:34.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:34 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:49:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:35.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:35 smithi175 ceph-mon[33399]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:49:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:35 smithi131 ceph-mon[29369]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:35 smithi131 ceph-mon[40885]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:37 smithi131 ceph-mon[29369]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:37 smithi131 ceph-mon[40885]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:37 smithi175 ceph-mon[33399]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-94"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-94"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-94"}]: dispatch 2024-09-10T18:49:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:38 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-94"}]: dispatch 2024-09-10T18:49:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-94"}]: dispatch 2024-09-10T18:49:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:38 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-94"}]: dispatch 2024-09-10T18:49:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:39 smithi131 ceph-mon[29369]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:49:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:39 smithi131 ceph-mon[29369]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T18:49:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:39 smithi131 ceph-mon[40885]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:49:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:39 smithi131 ceph-mon[40885]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T18:49:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:39 smithi175 ceph-mon[33399]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:49:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:39 smithi175 ceph-mon[33399]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[29369]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[29369]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[40885]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:40 smithi131 ceph-mon[40885]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T18:49:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:40 smithi175 ceph-mon[33399]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T18:49:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:40 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:40 smithi175 ceph-mon[33399]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T18:49:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[29369]: pgmap v741: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[29369]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T18:49:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[40885]: pgmap v741: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:41 smithi131 ceph-mon[40885]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T18:49:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:41 smithi175 ceph-mon[33399]: pgmap v741: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:41 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:49:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:41 smithi175 ceph-mon[33399]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T18:49:43.755 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:43 smithi175 ceph-mon[33399]: pgmap v744: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:43.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:43 smithi131 ceph-mon[29369]: pgmap v744: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:43 smithi131 ceph-mon[40885]: pgmap v744: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:49:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:49:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:43] "GET /metrics HTTP/1.1" 200 228521 "" "Prometheus/2.43.0" 2024-09-10T18:49:44.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:49:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:49:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:49:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:45 smithi131 ceph-mon[29369]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 861 B/s rd, 861 B/s wr, 2 op/s 2024-09-10T18:49:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:45 smithi131 ceph-mon[40885]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 861 B/s rd, 861 B/s wr, 2 op/s 2024-09-10T18:49:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:45 smithi175 ceph-mon[33399]: pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 861 B/s rd, 861 B/s wr, 2 op/s 2024-09-10T18:49:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:49:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=sqlstore.transactions t=2024-09-10T18:49:47.173443051Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:49:47.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:49:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=cleanup t=2024-09-10T18:49:47.17463658Z level=info msg="Completed cleanup jobs" duration=12.543573ms 2024-09-10T18:49:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:47 smithi131 ceph-mon[29369]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 737 B/s wr, 2 op/s 2024-09-10T18:49:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:47 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:47 smithi131 ceph-mon[40885]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 737 B/s wr, 2 op/s 2024-09-10T18:49:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:47 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:47 smithi175 ceph-mon[33399]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 737 B/s rd, 737 B/s wr, 2 op/s 2024-09-10T18:49:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:47 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:49:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:49 smithi131 ceph-mon[29369]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-10T18:49:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:49 smithi131 ceph-mon[40885]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-10T18:49:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:49 smithi175 ceph-mon[33399]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-09-10T18:49:51.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:51 smithi131 ceph-mon[29369]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 515 B/s wr, 2 op/s 2024-09-10T18:49:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:51 smithi131 ceph-mon[40885]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 515 B/s wr, 2 op/s 2024-09-10T18:49:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:51 smithi175 ceph-mon[33399]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 515 B/s wr, 2 op/s 2024-09-10T18:49:51.927 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:49:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:49:51.472+0000 7fb383132640 -1 snap_mapper.add_oid found existing snaps mapped on 64:b5d6c7ea:test-rados-api-smithi131-80444-97::foo:31, removing 2024-09-10T18:49:51.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:49:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:49:51.472+0000 7f38430b6640 -1 snap_mapper.add_oid found existing snaps mapped on 64:b5d6c7ea:test-rados-api-smithi131-80444-97::foo:31, removing 2024-09-10T18:49:51.928 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:49:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:49:51.472+0000 7fa21decf640 -1 snap_mapper.add_oid found existing snaps mapped on 64:b5d6c7ea:test-rados-api-smithi131-80444-97::foo:31, removing 2024-09-10T18:49:52.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-96"}]: dispatch 2024-09-10T18:49:52.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-96"}]: dispatch 2024-09-10T18:49:52.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:52.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:52.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-96"}]: dispatch 2024-09-10T18:49:52.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:52 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-96"}]: dispatch 2024-09-10T18:49:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:49:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-96"}]: dispatch 2024-09-10T18:49:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:52 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-96"}]: dispatch 2024-09-10T18:49:53.766 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:53 smithi175 ceph-mon[33399]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 468 B/s wr, 1 op/s 2024-09-10T18:49:53.767 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:53 smithi175 ceph-mon[33399]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T18:49:53.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:53 smithi131 ceph-mon[29369]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 468 B/s wr, 1 op/s 2024-09-10T18:49:53.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:53 smithi131 ceph-mon[29369]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T18:49:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:53 smithi131 ceph-mon[40885]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 937 B/s rd, 468 B/s wr, 1 op/s 2024-09-10T18:49:53.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:53 smithi131 ceph-mon[40885]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T18:49:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:49:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:53] "GET /metrics HTTP/1.1" 200 228516 "" "Prometheus/2.43.0" 2024-09-10T18:49:54.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:49:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[29369]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[40885]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:54.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:54 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:54 smithi175 ceph-mon[33399]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T18:49:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:54 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:49:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:54 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:49:55.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:49:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: pgmap v752: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]: dispatch 2024-09-10T18:49:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]: dispatch 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: pgmap v752: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]: dispatch 2024-09-10T18:49:55.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:55 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]: dispatch 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: pgmap v752: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]: dispatch 2024-09-10T18:49:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:55 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]: dispatch 2024-09-10T18:49:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]': finished 2024-09-10T18:49:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[29369]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T18:49:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:49:56.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:49:56.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:56.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]': finished 2024-09-10T18:49:56.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[40885]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T18:49:56.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:49:56.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:56 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:49:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:56 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:49:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:56 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-98", "mode": "writeback"}]': finished 2024-09-10T18:49:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:56 smithi175 ceph-mon[33399]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T18:49:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:49:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:56 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[29369]: pgmap v755: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]': finished 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[29369]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[40885]: pgmap v755: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]': finished 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[40885]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T18:49:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:49:57.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:57 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:49:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:57 smithi175 ceph-mon[33399]: pgmap v755: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:49:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-98"}]': finished 2024-09-10T18:49:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:57 smithi175 ceph-mon[33399]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T18:49:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:49:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:57 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[29369]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:58.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:49:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[40885]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T18:49:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:49:58.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:58 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:49:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:58 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:49:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:49:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:58 smithi175 ceph-mon[33399]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T18:49:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:49:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:58 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[29369]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[29369]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[40885]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[40885]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T18:49:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:49:59.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:49:59 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:49:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:59 smithi175 ceph-mon[33399]: pgmap v758: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:49:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T18:49:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:59 smithi175 ceph-mon[33399]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T18:49:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:49:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:49:59 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:50:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:50:00.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:00 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:50:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:00 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T18:50:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:01 smithi131 ceph-mon[29369]: pgmap v761: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:01 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T18:50:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:01 smithi131 ceph-mon[29369]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T18:50:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:01 smithi131 ceph-mon[40885]: pgmap v761: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:01 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T18:50:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:01 smithi131 ceph-mon[40885]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T18:50:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:01 smithi175 ceph-mon[33399]: pgmap v761: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:01 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T18:50:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:01 smithi175 ceph-mon[33399]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T18:50:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:02.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:02.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:02 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:02 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[29369]: pgmap v764: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:50:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[29369]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T18:50:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:50:03.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:50:03.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[40885]: pgmap v764: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:50:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[40885]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T18:50:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:50:03.806 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:50:03.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:50:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:50:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:03 smithi175 ceph-mon[33399]: pgmap v764: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:50:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:03 smithi175 ceph-mon[33399]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T18:50:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:50:03.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:03 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]: dispatch 2024-09-10T18:50:04.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:50:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:03] "GET /metrics HTTP/1.1" 200 228486 "" "Prometheus/2.43.0" 2024-09-10T18:50:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:04 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]': finished 2024-09-10T18:50:04.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:04 smithi131 ceph-mon[29369]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T18:50:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:04 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]': finished 2024-09-10T18:50:04.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:04 smithi131 ceph-mon[40885]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T18:50:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:04 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-98"}]': finished 2024-09-10T18:50:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:04 smithi175 ceph-mon[33399]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T18:50:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:50:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:05 smithi131 ceph-mon[29369]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:05.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:05 smithi131 ceph-mon[29369]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T18:50:05.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:05.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:05 smithi131 ceph-mon[40885]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:05.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:05 smithi131 ceph-mon[40885]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T18:50:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:05 smithi175 ceph-mon[33399]: pgmap v767: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:05 smithi175 ceph-mon[33399]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: osdmap e497: 8 total, 8 up, 8 in 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:06.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: osdmap e497: 8 total, 8 up, 8 in 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:06.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:06 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: osdmap e497: 8 total, 8 up, 8 in 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:06 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:50:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:07 smithi175 ceph-mon[33399]: pgmap v770: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:07 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:07.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:07 smithi175 ceph-mon[33399]: osdmap e498: 8 total, 8 up, 8 in 2024-09-10T18:50:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:07.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:07 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[29369]: pgmap v770: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[29369]: osdmap e498: 8 total, 8 up, 8 in 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[40885]: pgmap v770: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:50:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[40885]: osdmap e498: 8 total, 8 up, 8 in 2024-09-10T18:50:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:07 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[29369]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]': finished 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[29369]: osdmap e499: 8 total, 8 up, 8 in 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]: dispatch 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]: dispatch 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[40885]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]': finished 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[40885]: osdmap e499: 8 total, 8 up, 8 in 2024-09-10T18:50:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]: dispatch 2024-09-10T18:50:09.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:09 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]: dispatch 2024-09-10T18:50:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:09 smithi175 ceph-mon[33399]: pgmap v773: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-6", "overlaypool": "test-rados-api-smithi131-80444-100"}]': finished 2024-09-10T18:50:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:09 smithi175 ceph-mon[33399]: osdmap e499: 8 total, 8 up, 8 in 2024-09-10T18:50:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]: dispatch 2024-09-10T18:50:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:09 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]: dispatch 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]': finished 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[29369]: osdmap e500: 8 total, 8 up, 8 in 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]': finished 2024-09-10T18:50:10.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[40885]: osdmap e500: 8 total, 8 up, 8 in 2024-09-10T18:50:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:10.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:10 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:10 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-100", "mode": "writeback"}]': finished 2024-09-10T18:50:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:10 smithi175 ceph-mon[33399]: osdmap e500: 8 total, 8 up, 8 in 2024-09-10T18:50:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:10.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:10 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[29369]: pgmap v776: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[29369]: osdmap e501: 8 total, 8 up, 8 in 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[40885]: pgmap v776: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:50:11.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[40885]: osdmap e501: 8 total, 8 up, 8 in 2024-09-10T18:50:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:11.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:11 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:11 smithi175 ceph-mon[33399]: pgmap v776: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:50:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:11 smithi175 ceph-mon[33399]: osdmap e501: 8 total, 8 up, 8 in 2024-09-10T18:50:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:11 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:12 smithi175 ceph-mon[33399]: osdmap e502: 8 total, 8 up, 8 in 2024-09-10T18:50:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:12 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:12.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:12 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:13.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[29369]: osdmap e502: 8 total, 8 up, 8 in 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[40885]: osdmap e502: 8 total, 8 up, 8 in 2024-09-10T18:50:13.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:13.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:12 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-mon[33399]: pgmap v779: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:50:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-mon[33399]: osdmap e503: 8 total, 8 up, 8 in 2024-09-10T18:50:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:50:13.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:50:13.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:50:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:50:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:13] "GET /metrics HTTP/1.1" 200 228486 "" "Prometheus/2.43.0" 2024-09-10T18:50:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[29369]: pgmap v779: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:14.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[29369]: osdmap e503: 8 total, 8 up, 8 in 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[40885]: pgmap v779: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[40885]: osdmap e503: 8 total, 8 up, 8 in 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:50:14.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:13 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T18:50:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T18:50:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:14 smithi175 ceph-mon[33399]: osdmap e504: 8 total, 8 up, 8 in 2024-09-10T18:50:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:14.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:14 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T18:50:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[29369]: osdmap e504: 8 total, 8 up, 8 in 2024-09-10T18:50:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:15.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:15.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T18:50:15.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[40885]: osdmap e504: 8 total, 8 up, 8 in 2024-09-10T18:50:15.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:15.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:14 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-mon[33399]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:50:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-mon[33399]: osdmap e505: 8 total, 8 up, 8 in 2024-09-10T18:50:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:15.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:15 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[29369]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[29369]: osdmap e505: 8 total, 8 up, 8 in 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[40885]: pgmap v782: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:16.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]': finished 2024-09-10T18:50:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[40885]: osdmap e505: 8 total, 8 up, 8 in 2024-09-10T18:50:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:16.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:15 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]: dispatch 2024-09-10T18:50:16.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:16 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]': finished 2024-09-10T18:50:16.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:16 smithi175 ceph-mon[33399]: osdmap e506: 8 total, 8 up, 8 in 2024-09-10T18:50:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:16 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]': finished 2024-09-10T18:50:17.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:16 smithi131 ceph-mon[29369]: osdmap e506: 8 total, 8 up, 8 in 2024-09-10T18:50:17.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:16 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-100"}]': finished 2024-09-10T18:50:17.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:16 smithi131 ceph-mon[40885]: osdmap e506: 8 total, 8 up, 8 in 2024-09-10T18:50:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:17 smithi175 ceph-mon[33399]: pgmap v785: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:17 smithi175 ceph-mon[33399]: osdmap e507: 8 total, 8 up, 8 in 2024-09-10T18:50:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:18.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:17 smithi131 ceph-mon[29369]: pgmap v785: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:18.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:17 smithi131 ceph-mon[29369]: osdmap e507: 8 total, 8 up, 8 in 2024-09-10T18:50:18.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:18.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:17 smithi131 ceph-mon[40885]: pgmap v785: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:18.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:17 smithi131 ceph-mon[40885]: osdmap e507: 8 total, 8 up, 8 in 2024-09-10T18:50:18.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:18.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:18 smithi175 ceph-mon[33399]: osdmap e508: 8 total, 8 up, 8 in 2024-09-10T18:50:18.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:18.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:18 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:18 smithi131 ceph-mon[29369]: osdmap e508: 8 total, 8 up, 8 in 2024-09-10T18:50:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:19.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:18 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:19.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:18 smithi131 ceph-mon[40885]: osdmap e508: 8 total, 8 up, 8 in 2024-09-10T18:50:19.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:19.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:18 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: pgmap v788: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: osdmap e509: 8 total, 8 up, 8 in 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:50:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:19 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: pgmap v788: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: osdmap e509: 8 total, 8 up, 8 in 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:50:20.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: pgmap v788: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: osdmap e509: 8 total, 8 up, 8 in 2024-09-10T18:50:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T18:50:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:50:20.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:19 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T18:50:20.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:50:20.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:20 smithi175 ceph-mon[33399]: osdmap e510: 8 total, 8 up, 8 in 2024-09-10T18:50:20.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:50:20.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:20 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:50:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:50:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[29369]: osdmap e510: 8 total, 8 up, 8 in 2024-09-10T18:50:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:50:21.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:50:21.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T18:50:21.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[40885]: osdmap e510: 8 total, 8 up, 8 in 2024-09-10T18:50:21.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:50:21.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:20 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T18:50:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:21 smithi175 ceph-mon[33399]: pgmap v791: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:50:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:21 smithi175 ceph-mon[33399]: osdmap e511: 8 total, 8 up, 8 in 2024-09-10T18:50:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:50:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:21 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[29369]: pgmap v791: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[29369]: osdmap e511: 8 total, 8 up, 8 in 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[40885]: pgmap v791: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:22.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T18:50:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[40885]: osdmap e511: 8 total, 8 up, 8 in 2024-09-10T18:50:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:50:22.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:21 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T18:50:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:50:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:22 smithi175 ceph-mon[33399]: osdmap e512: 8 total, 8 up, 8 in 2024-09-10T18:50:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-102"}]: dispatch 2024-09-10T18:50:22.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:22 smithi175 ceph-mon[33399]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-102"}]: dispatch 2024-09-10T18:50:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:50:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[29369]: osdmap e512: 8 total, 8 up, 8 in 2024-09-10T18:50:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-102"}]: dispatch 2024-09-10T18:50:23.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[29369]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-102"}]: dispatch 2024-09-10T18:50:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T18:50:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[40885]: osdmap e512: 8 total, 8 up, 8 in 2024-09-10T18:50:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-6"}]: dispatch 2024-09-10T18:50:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1893972030' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-102"}]: dispatch 2024-09-10T18:50:23.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:22 smithi131 ceph-mon[40885]: from='client.25309 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-6", "tierpool": "test-rados-api-smithi131-80444-102"}]: dispatch 2024-09-10T18:50:23.927 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:50:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:50:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:23 smithi175 ceph-mon[33399]: pgmap v794: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:23.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:23 smithi175 ceph-mon[33399]: osdmap e513: 8 total, 8 up, 8 in 2024-09-10T18:50:24.076 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:23 smithi131 ceph-mon[40885]: pgmap v794: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:24.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:23 smithi131 ceph-mon[40885]: osdmap e513: 8 total, 8 up, 8 in 2024-09-10T18:50:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:23 smithi131 ceph-mon[29369]: pgmap v794: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:23 smithi131 ceph-mon[29369]: osdmap e513: 8 total, 8 up, 8 in 2024-09-10T18:50:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:50:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:23] "GET /metrics HTTP/1.1" 200 226072 "" "Prometheus/2.43.0" 2024-09-10T18:50:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:24 smithi175 ceph-mon[33399]: osdmap e514: 8 total, 8 up, 8 in 2024-09-10T18:50:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[29369]: osdmap e514: 8 total, 8 up, 8 in 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[40885]: osdmap e514: 8 total, 8 up, 8 in 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:25.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:50:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:50:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:25 smithi175 ceph-mon[33399]: pgmap v797: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:50:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:25 smithi175 ceph-mon[33399]: osdmap e515: 8 total, 8 up, 8 in 2024-09-10T18:50:26.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-80444-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:26.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[29369]: pgmap v797: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[29369]: osdmap e515: 8 total, 8 up, 8 in 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-80444-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[40885]: pgmap v797: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[40885]: osdmap e515: 8 total, 8 up, 8 in 2024-09-10T18:50:26.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-80444-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:27.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:26 smithi175 ceph-mon[33399]: osdmap e516: 8 total, 8 up, 8 in 2024-09-10T18:50:27.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:26 smithi175 ceph-mon[33399]: pgmap v800: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:27.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-80444-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:50:27.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:26 smithi175 ceph-mon[33399]: osdmap e517: 8 total, 8 up, 8 in 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[29369]: osdmap e516: 8 total, 8 up, 8 in 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[29369]: pgmap v800: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-80444-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[29369]: osdmap e517: 8 total, 8 up, 8 in 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[40885]: osdmap e516: 8 total, 8 up, 8 in 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[40885]: pgmap v800: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi131-80444-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:50:27.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:26 smithi131 ceph-mon[40885]: osdmap e517: 8 total, 8 up, 8 in 2024-09-10T18:50:28.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:27 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:28.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:27 smithi131 ceph-mon[40885]: osdmap e518: 8 total, 8 up, 8 in 2024-09-10T18:50:28.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:27 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:28.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:27 smithi131 ceph-mon[29369]: osdmap e518: 8 total, 8 up, 8 in 2024-09-10T18:50:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:27 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:28.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:27 smithi175 ceph-mon[33399]: osdmap e518: 8 total, 8 up, 8 in 2024-09-10T18:50:30.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:29 smithi131 ceph-mon[29369]: pgmap v803: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:29 smithi131 ceph-mon[29369]: osdmap e519: 8 total, 8 up, 8 in 2024-09-10T18:50:30.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:29 smithi131 ceph-mon[40885]: pgmap v803: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:29 smithi131 ceph-mon[40885]: osdmap e519: 8 total, 8 up, 8 in 2024-09-10T18:50:30.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:30.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:29 smithi175 ceph-mon[33399]: pgmap v803: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:29 smithi175 ceph-mon[33399]: osdmap e519: 8 total, 8 up, 8 in 2024-09-10T18:50:30.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[29369]: osdmap e520: 8 total, 8 up, 8 in 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[29369]: pgmap v806: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[29369]: osdmap e521: 8 total, 8 up, 8 in 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:31.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[40885]: osdmap e520: 8 total, 8 up, 8 in 2024-09-10T18:50:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[40885]: pgmap v806: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[40885]: osdmap e521: 8 total, 8 up, 8 in 2024-09-10T18:50:31.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:30 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:30 smithi175 ceph-mon[33399]: osdmap e520: 8 total, 8 up, 8 in 2024-09-10T18:50:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:30 smithi175 ceph-mon[33399]: pgmap v806: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:50:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:30 smithi175 ceph-mon[33399]: osdmap e521: 8 total, 8 up, 8 in 2024-09-10T18:50:31.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:30 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[29369]: osdmap e522: 8 total, 8 up, 8 in 2024-09-10T18:50:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:33.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[40885]: osdmap e522: 8 total, 8 up, 8 in 2024-09-10T18:50:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:33.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:32 smithi175 ceph-mon[33399]: osdmap e522: 8 total, 8 up, 8 in 2024-09-10T18:50:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:32 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:33.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:33.971 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:50:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:50:33.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:33 smithi175 ceph-mon[33399]: pgmap v809: 236 pgs: 64 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:33.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:33.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:33 smithi175 ceph-mon[33399]: osdmap e523: 8 total, 8 up, 8 in 2024-09-10T18:50:33.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-107", "overlaypool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:33.972 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:33.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[29369]: pgmap v809: 236 pgs: 64 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:33.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:33.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[29369]: osdmap e523: 8 total, 8 up, 8 in 2024-09-10T18:50:33.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-107", "overlaypool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:33.986 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:33.987 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:33] "GET /metrics HTTP/1.1" 200 230977 "" "Prometheus/2.43.0" 2024-09-10T18:50:33.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[40885]: pgmap v809: 236 pgs: 64 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:33.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:33.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[40885]: osdmap e523: 8 total, 8 up, 8 in 2024-09-10T18:50:33.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-107", "overlaypool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:33.987 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:35.038 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-107", "overlaypool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:35.038 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:34 smithi175 ceph-mon[33399]: osdmap e524: 8 total, 8 up, 8 in 2024-09-10T18:50:35.038 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:50:35.038 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:34 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:35.038 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:34 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-107-cache", "mode": "writeback"}]': finished 2024-09-10T18:50:35.038 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:34 smithi175 ceph-mon[33399]: osdmap e525: 8 total, 8 up, 8 in 2024-09-10T18:50:35.066 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-107", "overlaypool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:35.066 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[40885]: osdmap e524: 8 total, 8 up, 8 in 2024-09-10T18:50:35.066 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:50:35.066 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:35.066 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-107-cache", "mode": "writeback"}]': finished 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[40885]: osdmap e525: 8 total, 8 up, 8 in 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-107", "overlaypool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[29369]: osdmap e524: 8 total, 8 up, 8 in 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-107-cache", "mode": "writeback"}]': finished 2024-09-10T18:50:35.067 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:34 smithi131 ceph-mon[29369]: osdmap e525: 8 total, 8 up, 8 in 2024-09-10T18:50:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:50:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:35.962 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:35.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:35 smithi131 ceph-mon[29369]: pgmap v812: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:35.963 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:35 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-107"}]: dispatch 2024-09-10T18:50:35.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:35.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:35 smithi131 ceph-mon[40885]: pgmap v812: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:35.963 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:35 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-107"}]: dispatch 2024-09-10T18:50:36.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:35 smithi175 ceph-mon[33399]: pgmap v812: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:36.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:35 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-107"}]: dispatch 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-107"}]': finished 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[29369]: osdmap e526: 8 total, 8 up, 8 in 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-107"}]': finished 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[40885]: osdmap e526: 8 total, 8 up, 8 in 2024-09-10T18:50:37.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:37.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:37.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:36 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-107"}]': finished 2024-09-10T18:50:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:36 smithi175 ceph-mon[33399]: osdmap e526: 8 total, 8 up, 8 in 2024-09-10T18:50:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]: dispatch 2024-09-10T18:50:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:36 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:37.169 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:36 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: pgmap v815: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: osdmap e527: 8 total, 8 up, 8 in 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[29369]: osdmap e528: 8 total, 8 up, 8 in 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: pgmap v815: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: osdmap e527: 8 total, 8 up, 8 in 2024-09-10T18:50:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:37 smithi131 ceph-mon[40885]: osdmap e528: 8 total, 8 up, 8 in 2024-09-10T18:50:38.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: pgmap v815: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/217675031' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-107", "tierpool": "test-rados-api-smithi131-80444-107-cache"}]': finished 2024-09-10T18:50:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: osdmap e527: 8 total, 8 up, 8 in 2024-09-10T18:50:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:38.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:37 smithi175 ceph-mon[33399]: osdmap e528: 8 total, 8 up, 8 in 2024-09-10T18:50:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:39.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[29369]: osdmap e529: 8 total, 8 up, 8 in 2024-09-10T18:50:39.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:39.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:38 smithi131 ceph-mon[40885]: osdmap e529: 8 total, 8 up, 8 in 2024-09-10T18:50:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:39.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:38 smithi175 ceph-mon[33399]: osdmap e529: 8 total, 8 up, 8 in 2024-09-10T18:50:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[29369]: pgmap v818: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:40.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:50:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[40885]: pgmap v818: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:40.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:39 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:50:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:39 smithi175 ceph-mon[33399]: pgmap v818: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:40.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:39 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:50:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:50:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:40 smithi131 ceph-mon[29369]: osdmap e530: 8 total, 8 up, 8 in 2024-09-10T18:50:41.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:50:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:40 smithi131 ceph-mon[40885]: osdmap e530: 8 total, 8 up, 8 in 2024-09-10T18:50:41.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:41.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:50:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:40 smithi175 ceph-mon[33399]: osdmap e530: 8 total, 8 up, 8 in 2024-09-10T18:50:41.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[29369]: pgmap v821: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[29369]: osdmap e531: 8 total, 8 up, 8 in 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[29369]: osdmap e532: 8 total, 8 up, 8 in 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[40885]: pgmap v821: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[40885]: osdmap e531: 8 total, 8 up, 8 in 2024-09-10T18:50:42.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:42.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:41 smithi131 ceph-mon[40885]: osdmap e532: 8 total, 8 up, 8 in 2024-09-10T18:50:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:42 smithi175 ceph-mon[33399]: pgmap v821: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:42 smithi175 ceph-mon[33399]: osdmap e531: 8 total, 8 up, 8 in 2024-09-10T18:50:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:42.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:42 smithi175 ceph-mon[33399]: osdmap e532: 8 total, 8 up, 8 in 2024-09-10T18:50:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[29369]: pgmap v824: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[29369]: osdmap e533: 8 total, 8 up, 8 in 2024-09-10T18:50:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:43] "GET /metrics HTTP/1.1" 200 230977 "" "Prometheus/2.43.0" 2024-09-10T18:50:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[40885]: pgmap v824: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[40885]: osdmap e533: 8 total, 8 up, 8 in 2024-09-10T18:50:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:43 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:43 smithi175 ceph-mon[33399]: pgmap v824: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:43 smithi175 ceph-mon[33399]: osdmap e533: 8 total, 8 up, 8 in 2024-09-10T18:50:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:50:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:43 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:44.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:50:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: osdmap e534: 8 total, 8 up, 8 in 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: pgmap v827: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: osdmap e535: 8 total, 8 up, 8 in 2024-09-10T18:50:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-109", "overlaypool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: osdmap e534: 8 total, 8 up, 8 in 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: pgmap v827: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: osdmap e535: 8 total, 8 up, 8 in 2024-09-10T18:50:45.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-109", "overlaypool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: osdmap e534: 8 total, 8 up, 8 in 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: pgmap v827: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: osdmap e535: 8 total, 8 up, 8 in 2024-09-10T18:50:45.079 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-109", "overlaypool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:45.345 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:50:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-109", "overlaypool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:46 smithi131 ceph-mon[29369]: osdmap e536: 8 total, 8 up, 8 in 2024-09-10T18:50:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:50:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-109", "overlaypool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:46 smithi131 ceph-mon[40885]: osdmap e536: 8 total, 8 up, 8 in 2024-09-10T18:50:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:50:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-109", "overlaypool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:46 smithi175 ceph-mon[33399]: osdmap e536: 8 total, 8 up, 8 in 2024-09-10T18:50:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: pgmap v830: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-109-cache", "mode": "writeback"}]': finished 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: osdmap e537: 8 total, 8 up, 8 in 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: pgmap v830: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-109-cache", "mode": "writeback"}]': finished 2024-09-10T18:50:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: osdmap e537: 8 total, 8 up, 8 in 2024-09-10T18:50:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: pgmap v830: 212 pgs: 32 creating+peering, 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-109-cache", "mode": "writeback"}]': finished 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: osdmap e537: 8 total, 8 up, 8 in 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:50:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: osdmap e538: 8 total, 8 up, 8 in 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: pgmap v833: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: osdmap e539: 8 total, 8 up, 8 in 2024-09-10T18:50:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: osdmap e538: 8 total, 8 up, 8 in 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: pgmap v833: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: osdmap e539: 8 total, 8 up, 8 in 2024-09-10T18:50:49.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: osdmap e538: 8 total, 8 up, 8 in 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: pgmap v833: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: osdmap e539: 8 total, 8 up, 8 in 2024-09-10T18:50:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[29369]: osdmap e540: 8 total, 8 up, 8 in 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[40885]: osdmap e540: 8 total, 8 up, 8 in 2024-09-10T18:50:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:50:50.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:49 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:50:50.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:50:50.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:49 smithi175 ceph-mon[33399]: osdmap e540: 8 total, 8 up, 8 in 2024-09-10T18:50:50.179 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T18:50:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:50 smithi131 ceph-mon[29369]: pgmap v836: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T18:50:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:50 smithi131 ceph-mon[29369]: osdmap e541: 8 total, 8 up, 8 in 2024-09-10T18:50:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:50 smithi131 ceph-mon[40885]: pgmap v836: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T18:50:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:50 smithi131 ceph-mon[40885]: osdmap e541: 8 total, 8 up, 8 in 2024-09-10T18:50:51.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:50 smithi175 ceph-mon[33399]: pgmap v836: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:51.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T18:50:51.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:50 smithi175 ceph-mon[33399]: osdmap e541: 8 total, 8 up, 8 in 2024-09-10T18:50:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:52.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:53.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[29369]: osdmap e542: 8 total, 8 up, 8 in 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[29369]: pgmap v839: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[40885]: osdmap e542: 8 total, 8 up, 8 in 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:52 smithi131 ceph-mon[40885]: pgmap v839: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:52 smithi175 ceph-mon[33399]: osdmap e542: 8 total, 8 up, 8 in 2024-09-10T18:50:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]: dispatch 2024-09-10T18:50:53.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:52 smithi175 ceph-mon[33399]: pgmap v839: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:50:54.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:53 smithi131 ceph-mon[29369]: osdmap e543: 8 total, 8 up, 8 in 2024-09-10T18:50:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:50:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:53] "GET /metrics HTTP/1.1" 200 230967 "" "Prometheus/2.43.0" 2024-09-10T18:50:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:53 smithi131 ceph-mon[40885]: osdmap e543: 8 total, 8 up, 8 in 2024-09-10T18:50:54.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:50:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:50:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-109", "tierpool": "test-rados-api-smithi131-80444-109-cache"}]': finished 2024-09-10T18:50:54.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:53 smithi175 ceph-mon[33399]: osdmap e543: 8 total, 8 up, 8 in 2024-09-10T18:50:55.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:54 smithi131 ceph-mon[29369]: osdmap e544: 8 total, 8 up, 8 in 2024-09-10T18:50:55.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:54 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:55.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:54 smithi131 ceph-mon[29369]: pgmap v842: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:54 smithi131 ceph-mon[40885]: osdmap e544: 8 total, 8 up, 8 in 2024-09-10T18:50:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:54 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:54 smithi131 ceph-mon[40885]: pgmap v842: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:55.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:54 smithi175 ceph-mon[33399]: osdmap e544: 8 total, 8 up, 8 in 2024-09-10T18:50:55.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:54 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:50:55.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:54 smithi175 ceph-mon[33399]: pgmap v842: 180 pgs: 180 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:50:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:50:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:50:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:50:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:55 smithi131 ceph-mon[29369]: osdmap e545: 8 total, 8 up, 8 in 2024-09-10T18:50:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:55 smithi131 ceph-mon[40885]: osdmap e545: 8 total, 8 up, 8 in 2024-09-10T18:50:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:55 smithi175 ceph-mon[33399]: osdmap e545: 8 total, 8 up, 8 in 2024-09-10T18:50:56.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:57.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[29369]: osdmap e546: 8 total, 8 up, 8 in 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[29369]: pgmap v845: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[29369]: osdmap e547: 8 total, 8 up, 8 in 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[40885]: osdmap e546: 8 total, 8 up, 8 in 2024-09-10T18:50:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:57.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[40885]: pgmap v845: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:57.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:57.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:56 smithi131 ceph-mon[40885]: osdmap e547: 8 total, 8 up, 8 in 2024-09-10T18:50:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:56 smithi175 ceph-mon[33399]: osdmap e546: 8 total, 8 up, 8 in 2024-09-10T18:50:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]: dispatch 2024-09-10T18:50:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:56 smithi175 ceph-mon[33399]: pgmap v845: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:50:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/768684744' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-109"}]': finished 2024-09-10T18:50:57.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:56 smithi175 ceph-mon[33399]: osdmap e547: 8 total, 8 up, 8 in 2024-09-10T18:50:58.076 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:57 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:57 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:57 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:50:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:58 smithi175 ceph-mon[33399]: osdmap e548: 8 total, 8 up, 8 in 2024-09-10T18:50:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:58 smithi175 ceph-mon[33399]: pgmap v848: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:50:59.326 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:58 smithi131 ceph-mon[40885]: osdmap e548: 8 total, 8 up, 8 in 2024-09-10T18:50:59.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:59.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:58 smithi131 ceph-mon[40885]: pgmap v848: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:50:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:58 smithi131 ceph-mon[29369]: osdmap e548: 8 total, 8 up, 8 in 2024-09-10T18:50:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:50:59.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:58 smithi131 ceph-mon[29369]: pgmap v848: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:51:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:51:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:59 smithi175 ceph-mon[33399]: osdmap e549: 8 total, 8 up, 8 in 2024-09-10T18:51:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:50:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:51:00.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:51:00.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:59 smithi131 ceph-mon[29369]: osdmap e549: 8 total, 8 up, 8 in 2024-09-10T18:51:00.228 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:50:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:51:00.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:51:00.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:59 smithi131 ceph-mon[40885]: osdmap e549: 8 total, 8 up, 8 in 2024-09-10T18:51:00.229 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:50:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]: dispatch 2024-09-10T18:51:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:51:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:00 smithi175 ceph-mon[33399]: osdmap e550: 8 total, 8 up, 8 in 2024-09-10T18:51:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:51:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:00 smithi175 ceph-mon[33399]: pgmap v851: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:51:01.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:51:01.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[29369]: osdmap e550: 8 total, 8 up, 8 in 2024-09-10T18:51:01.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:01.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:01.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:51:01.241 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[29369]: pgmap v851: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:51:01.242 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/854703520' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi131-80444-104"}]': finished 2024-09-10T18:51:01.242 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[40885]: osdmap e550: 8 total, 8 up, 8 in 2024-09-10T18:51:01.242 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:01.242 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:01.242 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:51:01.242 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:00 smithi131 ceph-mon[40885]: pgmap v851: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:51:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:51:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:01 smithi175 ceph-mon[33399]: osdmap e551: 8 total, 8 up, 8 in 2024-09-10T18:51:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:02.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:51:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:01 smithi131 ceph-mon[29369]: osdmap e551: 8 total, 8 up, 8 in 2024-09-10T18:51:02.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi131-80444-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:51:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:01 smithi131 ceph-mon[40885]: osdmap e551: 8 total, 8 up, 8 in 2024-09-10T18:51:02.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:02 smithi175 ceph-mon[33399]: osdmap e552: 8 total, 8 up, 8 in 2024-09-10T18:51:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:02 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:02 smithi175 ceph-mon[33399]: pgmap v854: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:03.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[29369]: osdmap e552: 8 total, 8 up, 8 in 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[29369]: pgmap v854: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[40885]: osdmap e552: 8 total, 8 up, 8 in 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:03.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:02 smithi131 ceph-mon[40885]: pgmap v854: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:03 smithi131 ceph-mon[29369]: osdmap e553: 8 total, 8 up, 8 in 2024-09-10T18:51:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:51:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:03] "GET /metrics HTTP/1.1" 200 223691 "" "Prometheus/2.43.0" 2024-09-10T18:51:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:03 smithi131 ceph-mon[40885]: osdmap e553: 8 total, 8 up, 8 in 2024-09-10T18:51:04.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi131-80444-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:04.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:03 smithi175 ceph-mon[33399]: osdmap e553: 8 total, 8 up, 8 in 2024-09-10T18:51:04.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:51:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:51:05.177 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:04 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:04 smithi175 ceph-mon[33399]: osdmap e554: 8 total, 8 up, 8 in 2024-09-10T18:51:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:05.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:04 smithi175 ceph-mon[33399]: pgmap v857: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:05.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:05.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[29369]: osdmap e554: 8 total, 8 up, 8 in 2024-09-10T18:51:05.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:05.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[29369]: pgmap v857: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:05.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:05.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[40885]: osdmap e554: 8 total, 8 up, 8 in 2024-09-10T18:51:05.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:05.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:04 smithi131 ceph-mon[40885]: pgmap v857: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-mon[33399]: osdmap e555: 8 total, 8 up, 8 in 2024-09-10T18:51:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-mon[33399]: osdmap e556: 8 total, 8 up, 8 in 2024-09-10T18:51:06.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-112"}]: dispatch 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[29369]: osdmap e555: 8 total, 8 up, 8 in 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[29369]: osdmap e556: 8 total, 8 up, 8 in 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-112"}]: dispatch 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[40885]: osdmap e555: 8 total, 8 up, 8 in 2024-09-10T18:51:06.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:06.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:06.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[40885]: osdmap e556: 8 total, 8 up, 8 in 2024-09-10T18:51:06.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-112"}]: dispatch 2024-09-10T18:51:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:06 smithi175 ceph-mon[33399]: pgmap v860: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-112"}]': finished 2024-09-10T18:51:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:06 smithi175 ceph-mon[33399]: osdmap e557: 8 total, 8 up, 8 in 2024-09-10T18:51:07.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:06 smithi131 ceph-mon[29369]: pgmap v860: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-112"}]': finished 2024-09-10T18:51:07.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:06 smithi131 ceph-mon[29369]: osdmap e557: 8 total, 8 up, 8 in 2024-09-10T18:51:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:06 smithi131 ceph-mon[40885]: pgmap v860: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-112"}]': finished 2024-09-10T18:51:07.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:06 smithi131 ceph-mon[40885]: osdmap e557: 8 total, 8 up, 8 in 2024-09-10T18:51:08.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:07 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:08.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:08.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:07 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:08.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:07 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:08 smithi175 ceph-mon[33399]: osdmap e558: 8 total, 8 up, 8 in 2024-09-10T18:51:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112"}]: dispatch 2024-09-10T18:51:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:08 smithi175 ceph-mon[33399]: pgmap v863: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:51:09.326 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[40885]: osdmap e558: 8 total, 8 up, 8 in 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112"}]: dispatch 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[40885]: pgmap v863: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[29369]: osdmap e558: 8 total, 8 up, 8 in 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112"}]: dispatch 2024-09-10T18:51:09.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:08 smithi131 ceph-mon[29369]: pgmap v863: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:51:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112"}]': finished 2024-09-10T18:51:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:09 smithi175 ceph-mon[33399]: osdmap e559: 8 total, 8 up, 8 in 2024-09-10T18:51:10.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112"}]': finished 2024-09-10T18:51:10.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:09 smithi131 ceph-mon[29369]: osdmap e559: 8 total, 8 up, 8 in 2024-09-10T18:51:10.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-112"}]': finished 2024-09-10T18:51:10.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:09 smithi131 ceph-mon[40885]: osdmap e559: 8 total, 8 up, 8 in 2024-09-10T18:51:11.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:10 smithi175 ceph-mon[33399]: osdmap e560: 8 total, 8 up, 8 in 2024-09-10T18:51:11.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:10 smithi175 ceph-mon[33399]: pgmap v866: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:11.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:10 smithi131 ceph-mon[29369]: osdmap e560: 8 total, 8 up, 8 in 2024-09-10T18:51:11.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:10 smithi131 ceph-mon[29369]: pgmap v866: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:10 smithi131 ceph-mon[40885]: osdmap e560: 8 total, 8 up, 8 in 2024-09-10T18:51:11.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:10 smithi131 ceph-mon[40885]: pgmap v866: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:12.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:11 smithi131 ceph-mon[29369]: osdmap e561: 8 total, 8 up, 8 in 2024-09-10T18:51:12.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:12.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:11 smithi131 ceph-mon[40885]: osdmap e561: 8 total, 8 up, 8 in 2024-09-10T18:51:12.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:12.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:11 smithi175 ceph-mon[33399]: osdmap e561: 8 total, 8 up, 8 in 2024-09-10T18:51:12.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: osdmap e562: 8 total, 8 up, 8 in 2024-09-10T18:51:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: pgmap v869: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:13.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:13.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: osdmap e563: 8 total, 8 up, 8 in 2024-09-10T18:51:13.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-114"}]: dispatch 2024-09-10T18:51:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: osdmap e562: 8 total, 8 up, 8 in 2024-09-10T18:51:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:13.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: pgmap v869: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: osdmap e563: 8 total, 8 up, 8 in 2024-09-10T18:51:13.329 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-114"}]: dispatch 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: osdmap e562: 8 total, 8 up, 8 in 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: pgmap v869: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: osdmap e563: 8 total, 8 up, 8 in 2024-09-10T18:51:13.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-114"}]: dispatch 2024-09-10T18:51:14.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:51:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:13] "GET /metrics HTTP/1.1" 200 223691 "" "Prometheus/2.43.0" 2024-09-10T18:51:14.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:51:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:51:15.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-114"}]': finished 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[29369]: osdmap e564: 8 total, 8 up, 8 in 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-114", "mode": "writeback"}]: dispatch 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[29369]: pgmap v872: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-114"}]': finished 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[40885]: osdmap e564: 8 total, 8 up, 8 in 2024-09-10T18:51:15.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-114", "mode": "writeback"}]: dispatch 2024-09-10T18:51:15.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:14 smithi131 ceph-mon[40885]: pgmap v872: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:51:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:14 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-114"}]': finished 2024-09-10T18:51:15.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:14 smithi175 ceph-mon[33399]: osdmap e564: 8 total, 8 up, 8 in 2024-09-10T18:51:15.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-114", "mode": "writeback"}]: dispatch 2024-09-10T18:51:15.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:14 smithi175 ceph-mon[33399]: pgmap v872: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:16.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-114", "mode": "writeback"}]': finished 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[29369]: osdmap e565: 8 total, 8 up, 8 in 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-114", "mode": "writeback"}]': finished 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[40885]: osdmap e565: 8 total, 8 up, 8 in 2024-09-10T18:51:16.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:15 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-114", "mode": "writeback"}]': finished 2024-09-10T18:51:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:15 smithi175 ceph-mon[33399]: osdmap e565: 8 total, 8 up, 8 in 2024-09-10T18:51:16.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[29369]: osdmap e566: 8 total, 8 up, 8 in 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114"}]: dispatch 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[29369]: pgmap v875: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[40885]: osdmap e566: 8 total, 8 up, 8 in 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114"}]: dispatch 2024-09-10T18:51:17.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[40885]: pgmap v875: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:17.328 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:16 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:16 smithi175 ceph-mon[33399]: osdmap e566: 8 total, 8 up, 8 in 2024-09-10T18:51:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114"}]: dispatch 2024-09-10T18:51:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:16 smithi175 ceph-mon[33399]: pgmap v875: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:51:17.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:16 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:18.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114"}]': finished 2024-09-10T18:51:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:17 smithi131 ceph-mon[29369]: osdmap e567: 8 total, 8 up, 8 in 2024-09-10T18:51:18.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:18.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114"}]': finished 2024-09-10T18:51:18.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:17 smithi131 ceph-mon[40885]: osdmap e567: 8 total, 8 up, 8 in 2024-09-10T18:51:18.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-114"}]': finished 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:17 smithi175 ceph-mon[33399]: osdmap e567: 8 total, 8 up, 8 in 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:18 smithi131 ceph-mon[29369]: osdmap e568: 8 total, 8 up, 8 in 2024-09-10T18:51:19.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:18 smithi131 ceph-mon[29369]: pgmap v878: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:19.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:18 smithi131 ceph-mon[40885]: osdmap e568: 8 total, 8 up, 8 in 2024-09-10T18:51:19.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:18 smithi131 ceph-mon[40885]: pgmap v878: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:18 smithi175 ceph-mon[33399]: osdmap e568: 8 total, 8 up, 8 in 2024-09-10T18:51:19.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:18 smithi175 ceph-mon[33399]: pgmap v878: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:19 smithi131 ceph-mon[29369]: osdmap e569: 8 total, 8 up, 8 in 2024-09-10T18:51:20.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:20.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:19 smithi131 ceph-mon[40885]: osdmap e569: 8 total, 8 up, 8 in 2024-09-10T18:51:20.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:19 smithi175 ceph-mon[33399]: osdmap e569: 8 total, 8 up, 8 in 2024-09-10T18:51:20.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:21.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:20 smithi131 ceph-mon[29369]: osdmap e570: 8 total, 8 up, 8 in 2024-09-10T18:51:21.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:20 smithi131 ceph-mon[29369]: pgmap v881: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:21.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:21.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:20 smithi131 ceph-mon[40885]: osdmap e570: 8 total, 8 up, 8 in 2024-09-10T18:51:21.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:20 smithi131 ceph-mon[40885]: pgmap v881: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:21.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:20 smithi175 ceph-mon[33399]: osdmap e570: 8 total, 8 up, 8 in 2024-09-10T18:51:21.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:20 smithi175 ceph-mon[33399]: pgmap v881: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:21 smithi131 ceph-mon[29369]: osdmap e571: 8 total, 8 up, 8 in 2024-09-10T18:51:22.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:22.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:21 smithi131 ceph-mon[40885]: osdmap e571: 8 total, 8 up, 8 in 2024-09-10T18:51:22.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:22.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:21 smithi175 ceph-mon[33399]: osdmap e571: 8 total, 8 up, 8 in 2024-09-10T18:51:22.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:23.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[29369]: osdmap e572: 8 total, 8 up, 8 in 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-116"}]: dispatch 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[29369]: pgmap v884: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[40885]: osdmap e572: 8 total, 8 up, 8 in 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-116"}]: dispatch 2024-09-10T18:51:23.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:22 smithi131 ceph-mon[40885]: pgmap v884: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:22 smithi175 ceph-mon[33399]: osdmap e572: 8 total, 8 up, 8 in 2024-09-10T18:51:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-116"}]: dispatch 2024-09-10T18:51:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:22 smithi175 ceph-mon[33399]: pgmap v884: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:51:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:23] "GET /metrics HTTP/1.1" 200 228578 "" "Prometheus/2.43.0" 2024-09-10T18:51:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-116"}]': finished 2024-09-10T18:51:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:24 smithi131 ceph-mon[29369]: osdmap e573: 8 total, 8 up, 8 in 2024-09-10T18:51:24.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-116", "mode": "writeback"}]: dispatch 2024-09-10T18:51:24.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-116"}]': finished 2024-09-10T18:51:24.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:24 smithi131 ceph-mon[40885]: osdmap e573: 8 total, 8 up, 8 in 2024-09-10T18:51:24.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-116", "mode": "writeback"}]: dispatch 2024-09-10T18:51:24.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-116"}]': finished 2024-09-10T18:51:24.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:24 smithi175 ceph-mon[33399]: osdmap e573: 8 total, 8 up, 8 in 2024-09-10T18:51:24.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-116", "mode": "writeback"}]: dispatch 2024-09-10T18:51:24.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:51:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-116", "mode": "writeback"}]': finished 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: osdmap e574: 8 total, 8 up, 8 in 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.6"}]: dispatch 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.6"}]: dispatch 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[40885]: pgmap v887: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:51:25.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:25.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:25.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-116", "mode": "writeback"}]': finished 2024-09-10T18:51:25.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: osdmap e574: 8 total, 8 up, 8 in 2024-09-10T18:51:25.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.6"}]: dispatch 2024-09-10T18:51:25.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.6"}]: dispatch 2024-09-10T18:51:25.328 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:25 smithi131 ceph-mon[29369]: pgmap v887: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:51:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-116", "mode": "writeback"}]': finished 2024-09-10T18:51:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: osdmap e574: 8 total, 8 up, 8 in 2024-09-10T18:51:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.6"}]: dispatch 2024-09-10T18:51:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: from='mon.0 v1:172.21.15.131:0/1368249323' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.6"}]: dispatch 2024-09-10T18:51:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:25 smithi175 ceph-mon[33399]: pgmap v887: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:51:27.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:27 smithi131 ceph-mon[29369]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 944 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T18:51:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:27 smithi131 ceph-mon[40885]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 944 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T18:51:27.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:27 smithi175 ceph-mon[33399]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 944 B/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T18:51:29.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:29 smithi131 ceph-mon[29369]: pgmap v889: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-10T18:51:29.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:29 smithi131 ceph-mon[29369]: 244.6 scrub starts 2024-09-10T18:51:29.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:29 smithi131 ceph-mon[29369]: 244.6 scrub ok 2024-09-10T18:51:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:29 smithi131 ceph-mon[40885]: pgmap v889: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-10T18:51:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:29 smithi131 ceph-mon[40885]: 244.6 scrub starts 2024-09-10T18:51:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:29 smithi131 ceph-mon[40885]: 244.6 scrub ok 2024-09-10T18:51:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:29 smithi175 ceph-mon[33399]: pgmap v889: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-10T18:51:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:29 smithi175 ceph-mon[33399]: 244.6 scrub starts 2024-09-10T18:51:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:29 smithi175 ceph-mon[33399]: 244.6 scrub ok 2024-09-10T18:51:31.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:31 smithi131 ceph-mon[29369]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-09-10T18:51:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:31 smithi131 ceph-mon[40885]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-09-10T18:51:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:31 smithi175 ceph-mon[33399]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-09-10T18:51:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-mon[29369]: pgmap v891: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 761 B/s wr, 2 op/s 2024-09-10T18:51:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-mon[40885]: pgmap v891: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 761 B/s wr, 2 op/s 2024-09-10T18:51:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:33 smithi175 ceph-mon[33399]: pgmap v891: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 761 B/s wr, 2 op/s 2024-09-10T18:51:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:51:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:33] "GET /metrics HTTP/1.1" 200 228592 "" "Prometheus/2.43.0" 2024-09-10T18:51:34.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:51:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:51:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:51:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:35 smithi131 ceph-mon[29369]: pgmap v892: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:35 smithi131 ceph-mon[40885]: pgmap v892: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:35 smithi175 ceph-mon[33399]: pgmap v892: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:37 smithi131 ceph-mon[29369]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:51:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:37 smithi131 ceph-mon[40885]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:51:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:37 smithi175 ceph-mon[33399]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-09-10T18:51:38.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:38 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:38.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:38 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:39 smithi131 ceph-mon[29369]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:51:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:39 smithi131 ceph-mon[40885]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:51:39.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:39 smithi175 ceph-mon[33399]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:51:40.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:40 smithi131 ceph-mon[29369]: osdmap e575: 8 total, 8 up, 8 in 2024-09-10T18:51:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:40 smithi131 ceph-mon[40885]: osdmap e575: 8 total, 8 up, 8 in 2024-09-10T18:51:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:40.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:40 smithi175 ceph-mon[33399]: osdmap e575: 8 total, 8 up, 8 in 2024-09-10T18:51:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[29369]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[29369]: osdmap e576: 8 total, 8 up, 8 in 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116"}]: dispatch 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[40885]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[40885]: osdmap e576: 8 total, 8 up, 8 in 2024-09-10T18:51:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116"}]: dispatch 2024-09-10T18:51:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:41 smithi175 ceph-mon[33399]: pgmap v896: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:51:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:41 smithi175 ceph-mon[33399]: osdmap e576: 8 total, 8 up, 8 in 2024-09-10T18:51:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116"}]: dispatch 2024-09-10T18:51:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:42 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116"}]': finished 2024-09-10T18:51:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:42 smithi131 ceph-mon[29369]: osdmap e577: 8 total, 8 up, 8 in 2024-09-10T18:51:42.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:42 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:42.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116"}]': finished 2024-09-10T18:51:42.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:42 smithi131 ceph-mon[40885]: osdmap e577: 8 total, 8 up, 8 in 2024-09-10T18:51:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:42 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-116"}]': finished 2024-09-10T18:51:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:42 smithi175 ceph-mon[33399]: osdmap e577: 8 total, 8 up, 8 in 2024-09-10T18:51:43.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-mon[29369]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:43.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:43.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-mon[29369]: osdmap e578: 8 total, 8 up, 8 in 2024-09-10T18:51:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-mon[40885]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-mon[40885]: osdmap e578: 8 total, 8 up, 8 in 2024-09-10T18:51:43.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:43 smithi175 ceph-mon[33399]: pgmap v899: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:51:43.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:43 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:43.875 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:43 smithi175 ceph-mon[33399]: osdmap e578: 8 total, 8 up, 8 in 2024-09-10T18:51:44.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:51:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:43] "GET /metrics HTTP/1.1" 200 228592 "" "Prometheus/2.43.0" 2024-09-10T18:51:44.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:51:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:51:44.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:44 smithi131 ceph-mon[29369]: osdmap e579: 8 total, 8 up, 8 in 2024-09-10T18:51:44.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:44.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:44 smithi131 ceph-mon[40885]: osdmap e579: 8 total, 8 up, 8 in 2024-09-10T18:51:44.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:44.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:44 smithi175 ceph-mon[33399]: osdmap e579: 8 total, 8 up, 8 in 2024-09-10T18:51:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:51:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[29369]: pgmap v902: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[29369]: osdmap e580: 8 total, 8 up, 8 in 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[40885]: pgmap v902: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:45 smithi131 ceph-mon[40885]: osdmap e580: 8 total, 8 up, 8 in 2024-09-10T18:51:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:45 smithi175 ceph-mon[33399]: pgmap v902: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:51:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:45 smithi175 ceph-mon[33399]: osdmap e580: 8 total, 8 up, 8 in 2024-09-10T18:51:46.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:46 smithi131 ceph-mon[29369]: osdmap e581: 8 total, 8 up, 8 in 2024-09-10T18:51:46.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:46.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:46 smithi131 ceph-mon[40885]: osdmap e581: 8 total, 8 up, 8 in 2024-09-10T18:51:46.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:46 smithi175 ceph-mon[33399]: osdmap e581: 8 total, 8 up, 8 in 2024-09-10T18:51:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: pgmap v905: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: osdmap e582: 8 total, 8 up, 8 in 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-118"}]: dispatch 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-118"}]': finished 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: osdmap e583: 8 total, 8 up, 8 in 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-118", "mode": "writeback"}]: dispatch 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: pgmap v905: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: osdmap e582: 8 total, 8 up, 8 in 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-118"}]: dispatch 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-118"}]': finished 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: osdmap e583: 8 total, 8 up, 8 in 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-118", "mode": "writeback"}]: dispatch 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: pgmap v905: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: osdmap e582: 8 total, 8 up, 8 in 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-118"}]: dispatch 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-118"}]': finished 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: osdmap e583: 8 total, 8 up, 8 in 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-118", "mode": "writeback"}]: dispatch 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:51:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:51:48.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:48 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:48.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-118", "mode": "writeback"}]': finished 2024-09-10T18:51:48.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:48 smithi131 ceph-mon[29369]: osdmap e584: 8 total, 8 up, 8 in 2024-09-10T18:51:48.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:48 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:48.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-118", "mode": "writeback"}]': finished 2024-09-10T18:51:48.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:48 smithi131 ceph-mon[40885]: osdmap e584: 8 total, 8 up, 8 in 2024-09-10T18:51:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:48 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-118", "mode": "writeback"}]': finished 2024-09-10T18:51:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:48 smithi175 ceph-mon[33399]: osdmap e584: 8 total, 8 up, 8 in 2024-09-10T18:51:49.545 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:49 smithi131 ceph-mon[29369]: pgmap v909: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 526 B/s wr, 1 op/s 2024-09-10T18:51:49.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:49 smithi131 ceph-mon[29369]: osdmap e585: 8 total, 8 up, 8 in 2024-09-10T18:51:49.546 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:49 smithi131 ceph-mon[40885]: pgmap v909: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 526 B/s wr, 1 op/s 2024-09-10T18:51:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:49 smithi131 ceph-mon[40885]: osdmap e585: 8 total, 8 up, 8 in 2024-09-10T18:51:49.546 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:49 smithi175 ceph-mon[33399]: pgmap v909: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 526 B/s wr, 1 op/s 2024-09-10T18:51:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:49 smithi175 ceph-mon[33399]: osdmap e585: 8 total, 8 up, 8 in 2024-09-10T18:51:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:51 smithi175 ceph-mon[33399]: osdmap e586: 8 total, 8 up, 8 in 2024-09-10T18:51:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118"}]: dispatch 2024-09-10T18:51:51.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:51 smithi175 ceph-mon[33399]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 528 B/s wr, 1 op/s 2024-09-10T18:51:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[29369]: osdmap e586: 8 total, 8 up, 8 in 2024-09-10T18:51:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118"}]: dispatch 2024-09-10T18:51:51.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[29369]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 528 B/s wr, 1 op/s 2024-09-10T18:51:51.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[40885]: osdmap e586: 8 total, 8 up, 8 in 2024-09-10T18:51:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118"}]: dispatch 2024-09-10T18:51:51.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:51 smithi131 ceph-mon[40885]: pgmap v912: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 528 B/s wr, 1 op/s 2024-09-10T18:51:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:52 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118"}]': finished 2024-09-10T18:51:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:52 smithi175 ceph-mon[33399]: osdmap e587: 8 total, 8 up, 8 in 2024-09-10T18:51:52.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:52 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:52.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118"}]': finished 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[29369]: osdmap e587: 8 total, 8 up, 8 in 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-118"}]': finished 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[40885]: osdmap e587: 8 total, 8 up, 8 in 2024-09-10T18:51:52.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:52 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:51:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:53 smithi131 ceph-mon[29369]: osdmap e588: 8 total, 8 up, 8 in 2024-09-10T18:51:53.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:53 smithi131 ceph-mon[29369]: pgmap v915: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:53.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:53 smithi131 ceph-mon[40885]: osdmap e588: 8 total, 8 up, 8 in 2024-09-10T18:51:53.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:53 smithi131 ceph-mon[40885]: pgmap v915: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:53 smithi175 ceph-mon[33399]: osdmap e588: 8 total, 8 up, 8 in 2024-09-10T18:51:53.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:53 smithi175 ceph-mon[33399]: pgmap v915: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:51:54.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:51:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:53] "GET /metrics HTTP/1.1" 200 228549 "" "Prometheus/2.43.0" 2024-09-10T18:51:54.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:51:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:51:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:54 smithi131 ceph-mon[29369]: osdmap e589: 8 total, 8 up, 8 in 2024-09-10T18:51:54.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:54 smithi131 ceph-mon[40885]: osdmap e589: 8 total, 8 up, 8 in 2024-09-10T18:51:54.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:54 smithi175 ceph-mon[33399]: osdmap e589: 8 total, 8 up, 8 in 2024-09-10T18:51:54.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:51:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:51:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:51:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:55 smithi175 ceph-mon[33399]: osdmap e590: 8 total, 8 up, 8 in 2024-09-10T18:51:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:55.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:55 smithi175 ceph-mon[33399]: pgmap v918: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[29369]: osdmap e590: 8 total, 8 up, 8 in 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[29369]: pgmap v918: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:51:55.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[40885]: osdmap e590: 8 total, 8 up, 8 in 2024-09-10T18:51:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:51:55.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:55 smithi131 ceph-mon[40885]: pgmap v918: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:56.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:56.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:56 smithi131 ceph-mon[29369]: osdmap e591: 8 total, 8 up, 8 in 2024-09-10T18:51:56.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-120"}]: dispatch 2024-09-10T18:51:56.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:56.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:56 smithi131 ceph-mon[40885]: osdmap e591: 8 total, 8 up, 8 in 2024-09-10T18:51:56.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-120"}]: dispatch 2024-09-10T18:51:56.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:51:56.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:56 smithi175 ceph-mon[33399]: osdmap e591: 8 total, 8 up, 8 in 2024-09-10T18:51:56.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-120"}]: dispatch 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-120"}]': finished 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: osdmap e592: 8 total, 8 up, 8 in 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-120", "mode": "writeback"}]: dispatch 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: pgmap v921: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-120", "mode": "writeback"}]': finished 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[29369]: osdmap e593: 8 total, 8 up, 8 in 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-120"}]': finished 2024-09-10T18:51:57.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: osdmap e592: 8 total, 8 up, 8 in 2024-09-10T18:51:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-120", "mode": "writeback"}]: dispatch 2024-09-10T18:51:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: pgmap v921: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-120", "mode": "writeback"}]': finished 2024-09-10T18:51:57.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:57 smithi131 ceph-mon[40885]: osdmap e593: 8 total, 8 up, 8 in 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-120"}]': finished 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: osdmap e592: 8 total, 8 up, 8 in 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-120", "mode": "writeback"}]: dispatch 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: pgmap v921: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-120", "mode": "writeback"}]': finished 2024-09-10T18:51:57.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:57 smithi175 ceph-mon[33399]: osdmap e593: 8 total, 8 up, 8 in 2024-09-10T18:51:58.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:58.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:58.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[29369]: osdmap e594: 8 total, 8 up, 8 in 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120"}]: dispatch 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[29369]: pgmap v924: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[40885]: osdmap e594: 8 total, 8 up, 8 in 2024-09-10T18:51:59.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120"}]: dispatch 2024-09-10T18:51:59.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:51:59 smithi131 ceph-mon[40885]: pgmap v924: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:51:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:51:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:59 smithi175 ceph-mon[33399]: osdmap e594: 8 total, 8 up, 8 in 2024-09-10T18:51:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120"}]: dispatch 2024-09-10T18:51:59.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:51:59 smithi175 ceph-mon[33399]: pgmap v924: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:52:00.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:00 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120"}]': finished 2024-09-10T18:52:00.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:00 smithi131 ceph-mon[29369]: osdmap e595: 8 total, 8 up, 8 in 2024-09-10T18:52:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:00 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120"}]': finished 2024-09-10T18:52:00.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:00 smithi131 ceph-mon[40885]: osdmap e595: 8 total, 8 up, 8 in 2024-09-10T18:52:00.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:00 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-120"}]': finished 2024-09-10T18:52:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:00 smithi175 ceph-mon[33399]: osdmap e595: 8 total, 8 up, 8 in 2024-09-10T18:52:01.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[29369]: osdmap e596: 8 total, 8 up, 8 in 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[29369]: pgmap v927: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[29369]: osdmap e597: 8 total, 8 up, 8 in 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[40885]: osdmap e596: 8 total, 8 up, 8 in 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[40885]: pgmap v927: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[40885]: osdmap e597: 8 total, 8 up, 8 in 2024-09-10T18:52:01.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:01.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:01 smithi175 ceph-mon[33399]: osdmap e596: 8 total, 8 up, 8 in 2024-09-10T18:52:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:01 smithi175 ceph-mon[33399]: pgmap v927: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:01 smithi175 ceph-mon[33399]: osdmap e597: 8 total, 8 up, 8 in 2024-09-10T18:52:01.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[29369]: osdmap e598: 8 total, 8 up, 8 in 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[29369]: pgmap v930: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[40885]: osdmap e598: 8 total, 8 up, 8 in 2024-09-10T18:52:03.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:03.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-mon[40885]: pgmap v930: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:03.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:03 smithi175 ceph-mon[33399]: osdmap e598: 8 total, 8 up, 8 in 2024-09-10T18:52:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:03 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:03.679 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:03 smithi175 ceph-mon[33399]: pgmap v930: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:04.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:52:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:03] "GET /metrics HTTP/1.1" 200 228584 "" "Prometheus/2.43.0" 2024-09-10T18:52:04.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:52:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:52:04.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:04.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:04 smithi131 ceph-mon[29369]: osdmap e599: 8 total, 8 up, 8 in 2024-09-10T18:52:04.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-122"}]: dispatch 2024-09-10T18:52:04.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:04.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:04 smithi131 ceph-mon[40885]: osdmap e599: 8 total, 8 up, 8 in 2024-09-10T18:52:04.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-122"}]: dispatch 2024-09-10T18:52:04.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:04.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:04 smithi175 ceph-mon[33399]: osdmap e599: 8 total, 8 up, 8 in 2024-09-10T18:52:04.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-122"}]: dispatch 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-122"}]': finished 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: osdmap e600: 8 total, 8 up, 8 in 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-122", "mode": "writeback"}]: dispatch 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: pgmap v933: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-122", "mode": "writeback"}]': finished 2024-09-10T18:52:05.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:05 smithi175 ceph-mon[33399]: osdmap e601: 8 total, 8 up, 8 in 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-122"}]': finished 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: osdmap e600: 8 total, 8 up, 8 in 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-122", "mode": "writeback"}]: dispatch 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: pgmap v933: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-122", "mode": "writeback"}]': finished 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[29369]: osdmap e601: 8 total, 8 up, 8 in 2024-09-10T18:52:05.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-122"}]': finished 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: osdmap e600: 8 total, 8 up, 8 in 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-122", "mode": "writeback"}]: dispatch 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: pgmap v933: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-122", "mode": "writeback"}]': finished 2024-09-10T18:52:05.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:05 smithi131 ceph-mon[40885]: osdmap e601: 8 total, 8 up, 8 in 2024-09-10T18:52:06.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:06.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:06.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[29369]: osdmap e602: 8 total, 8 up, 8 in 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122"}]: dispatch 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[29369]: pgmap v936: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[40885]: osdmap e602: 8 total, 8 up, 8 in 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122"}]: dispatch 2024-09-10T18:52:07.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:07 smithi131 ceph-mon[40885]: pgmap v936: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:07 smithi175 ceph-mon[33399]: osdmap e602: 8 total, 8 up, 8 in 2024-09-10T18:52:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122"}]: dispatch 2024-09-10T18:52:07.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:07 smithi175 ceph-mon[33399]: pgmap v936: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:08.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:08 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122"}]': finished 2024-09-10T18:52:08.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:08 smithi131 ceph-mon[29369]: osdmap e603: 8 total, 8 up, 8 in 2024-09-10T18:52:08.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:08 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:08.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122"}]': finished 2024-09-10T18:52:08.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:08 smithi131 ceph-mon[40885]: osdmap e603: 8 total, 8 up, 8 in 2024-09-10T18:52:08.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:08 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-122"}]': finished 2024-09-10T18:52:08.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:08 smithi175 ceph-mon[33399]: osdmap e603: 8 total, 8 up, 8 in 2024-09-10T18:52:09.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[29369]: osdmap e604: 8 total, 8 up, 8 in 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[29369]: pgmap v939: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[29369]: osdmap e605: 8 total, 8 up, 8 in 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[40885]: osdmap e604: 8 total, 8 up, 8 in 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[40885]: pgmap v939: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[40885]: osdmap e605: 8 total, 8 up, 8 in 2024-09-10T18:52:09.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:09.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:09 smithi175 ceph-mon[33399]: osdmap e604: 8 total, 8 up, 8 in 2024-09-10T18:52:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:09 smithi175 ceph-mon[33399]: pgmap v939: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:09 smithi175 ceph-mon[33399]: osdmap e605: 8 total, 8 up, 8 in 2024-09-10T18:52:09.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:11.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:11 smithi131 ceph-mon[29369]: osdmap e606: 8 total, 8 up, 8 in 2024-09-10T18:52:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:11 smithi131 ceph-mon[29369]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:11 smithi131 ceph-mon[40885]: osdmap e606: 8 total, 8 up, 8 in 2024-09-10T18:52:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:11 smithi131 ceph-mon[40885]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:11 smithi175 ceph-mon[33399]: osdmap e606: 8 total, 8 up, 8 in 2024-09-10T18:52:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:11 smithi175 ceph-mon[33399]: pgmap v942: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:12 smithi131 ceph-mon[29369]: osdmap e607: 8 total, 8 up, 8 in 2024-09-10T18:52:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:12 smithi131 ceph-mon[40885]: osdmap e607: 8 total, 8 up, 8 in 2024-09-10T18:52:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:12 smithi175 ceph-mon[33399]: osdmap e607: 8 total, 8 up, 8 in 2024-09-10T18:52:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:13.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: osdmap e608: 8 total, 8 up, 8 in 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-124"}]: dispatch 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-124"}]': finished 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: osdmap e609: 8 total, 8 up, 8 in 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-124", "mode": "writeback"}]: dispatch 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: osdmap e608: 8 total, 8 up, 8 in 2024-09-10T18:52:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-124"}]: dispatch 2024-09-10T18:52:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-124"}]': finished 2024-09-10T18:52:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: osdmap e609: 8 total, 8 up, 8 in 2024-09-10T18:52:13.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-124", "mode": "writeback"}]: dispatch 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: osdmap e608: 8 total, 8 up, 8 in 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-124"}]: dispatch 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: pgmap v945: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-124"}]': finished 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: osdmap e609: 8 total, 8 up, 8 in 2024-09-10T18:52:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-124", "mode": "writeback"}]: dispatch 2024-09-10T18:52:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:52:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:13] "GET /metrics HTTP/1.1" 200 228584 "" "Prometheus/2.43.0" 2024-09-10T18:52:14.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:52:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:52:14.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:14 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-124", "mode": "writeback"}]': finished 2024-09-10T18:52:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:14 smithi131 ceph-mon[29369]: osdmap e610: 8 total, 8 up, 8 in 2024-09-10T18:52:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:14 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-124", "mode": "writeback"}]': finished 2024-09-10T18:52:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:14 smithi131 ceph-mon[40885]: osdmap e610: 8 total, 8 up, 8 in 2024-09-10T18:52:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:14 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-124", "mode": "writeback"}]': finished 2024-09-10T18:52:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:14 smithi175 ceph-mon[33399]: osdmap e610: 8 total, 8 up, 8 in 2024-09-10T18:52:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:15 smithi175 ceph-mon[33399]: pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:52:15.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:52:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:15.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:15 smithi131 ceph-mon[29369]: pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:52:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:15 smithi131 ceph-mon[40885]: pgmap v948: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:52:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:16 smithi131 ceph-mon[29369]: osdmap e611: 8 total, 8 up, 8 in 2024-09-10T18:52:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:16 smithi131 ceph-mon[40885]: osdmap e611: 8 total, 8 up, 8 in 2024-09-10T18:52:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:16.646 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:16 smithi175 ceph-mon[33399]: osdmap e611: 8 total, 8 up, 8 in 2024-09-10T18:52:16.646 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: osdmap e612: 8 total, 8 up, 8 in 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124"}]: dispatch 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: pgmap v951: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: osdmap e612: 8 total, 8 up, 8 in 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124"}]: dispatch 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: pgmap v951: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:17.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:17 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: osdmap e612: 8 total, 8 up, 8 in 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124"}]: dispatch 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: pgmap v951: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:17 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:18.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124"}]': finished 2024-09-10T18:52:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:18 smithi175 ceph-mon[33399]: osdmap e613: 8 total, 8 up, 8 in 2024-09-10T18:52:18.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124"}]': finished 2024-09-10T18:52:18.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:18 smithi131 ceph-mon[40885]: osdmap e613: 8 total, 8 up, 8 in 2024-09-10T18:52:18.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-124"}]': finished 2024-09-10T18:52:18.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:18 smithi131 ceph-mon[29369]: osdmap e613: 8 total, 8 up, 8 in 2024-09-10T18:52:19.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:19 smithi175 ceph-mon[33399]: osdmap e614: 8 total, 8 up, 8 in 2024-09-10T18:52:19.681 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:19 smithi175 ceph-mon[33399]: pgmap v954: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:19.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:19 smithi131 ceph-mon[40885]: osdmap e614: 8 total, 8 up, 8 in 2024-09-10T18:52:19.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:19 smithi131 ceph-mon[40885]: pgmap v954: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:19.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:19 smithi131 ceph-mon[29369]: osdmap e614: 8 total, 8 up, 8 in 2024-09-10T18:52:19.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:19 smithi131 ceph-mon[29369]: pgmap v954: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:20 smithi175 ceph-mon[33399]: osdmap e615: 8 total, 8 up, 8 in 2024-09-10T18:52:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:20.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:20 smithi131 ceph-mon[29369]: osdmap e615: 8 total, 8 up, 8 in 2024-09-10T18:52:20.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:20.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:20 smithi131 ceph-mon[40885]: osdmap e615: 8 total, 8 up, 8 in 2024-09-10T18:52:20.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: osdmap e616: 8 total, 8 up, 8 in 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: osdmap e617: 8 total, 8 up, 8 in 2024-09-10T18:52:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-126"}]: dispatch 2024-09-10T18:52:21.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: osdmap e616: 8 total, 8 up, 8 in 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: osdmap e617: 8 total, 8 up, 8 in 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-126"}]: dispatch 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: osdmap e616: 8 total, 8 up, 8 in 2024-09-10T18:52:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: pgmap v957: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: osdmap e617: 8 total, 8 up, 8 in 2024-09-10T18:52:21.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-126"}]: dispatch 2024-09-10T18:52:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-126"}]': finished 2024-09-10T18:52:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:23 smithi175 ceph-mon[33399]: osdmap e618: 8 total, 8 up, 8 in 2024-09-10T18:52:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-126", "mode": "writeback"}]: dispatch 2024-09-10T18:52:23.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:23 smithi175 ceph-mon[33399]: pgmap v960: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:23.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-126"}]': finished 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[29369]: osdmap e618: 8 total, 8 up, 8 in 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-126", "mode": "writeback"}]: dispatch 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[29369]: pgmap v960: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-126"}]': finished 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[40885]: osdmap e618: 8 total, 8 up, 8 in 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-126", "mode": "writeback"}]: dispatch 2024-09-10T18:52:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-mon[40885]: pgmap v960: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:24.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:52:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:23] "GET /metrics HTTP/1.1" 200 228574 "" "Prometheus/2.43.0" 2024-09-10T18:52:24.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:52:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:52:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-126", "mode": "writeback"}]': finished 2024-09-10T18:52:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[29369]: osdmap e619: 8 total, 8 up, 8 in 2024-09-10T18:52:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:24.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-126", "mode": "writeback"}]': finished 2024-09-10T18:52:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[40885]: osdmap e619: 8 total, 8 up, 8 in 2024-09-10T18:52:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:24 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-126", "mode": "writeback"}]': finished 2024-09-10T18:52:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:24 smithi175 ceph-mon[33399]: osdmap e619: 8 total, 8 up, 8 in 2024-09-10T18:52:24.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: osdmap e620: 8 total, 8 up, 8 in 2024-09-10T18:52:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126"}]: dispatch 2024-09-10T18:52:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: pgmap v963: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126"}]': finished 2024-09-10T18:52:25.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:25 smithi175 ceph-mon[33399]: osdmap e621: 8 total, 8 up, 8 in 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: osdmap e620: 8 total, 8 up, 8 in 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126"}]: dispatch 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: pgmap v963: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126"}]': finished 2024-09-10T18:52:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[29369]: osdmap e621: 8 total, 8 up, 8 in 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: osdmap e620: 8 total, 8 up, 8 in 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126"}]: dispatch 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: pgmap v963: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-126"}]': finished 2024-09-10T18:52:25.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:25 smithi131 ceph-mon[40885]: osdmap e621: 8 total, 8 up, 8 in 2024-09-10T18:52:26.189 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-10T18:52:26.189 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:52:26.189 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9033 ms) 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6034 ms) 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5029 ms) 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:52:26.190 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25505 ms) 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14048 ms) 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12066 ms) 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 00000012 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12063 ms) 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6047 ms) 2024-09-10T18:52:26.191 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (553727 ms total) 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1011 ms) 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11034 ms) 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18089 ms) 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30136 ms total) 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-09-10T18:52:26.192 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7053 ms) 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8041 ms) 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: waiting for scrub... 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: done waiting 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24547 ms) 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9668 ms) 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8063 ms) 2024-09-10T18:52:26.193 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-10T18:52:26.194 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8035 ms) 2024-09-10T18:52:26.194 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-10T18:52:26.194 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10058 ms) 2024-09-10T18:52:26.194 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-10T18:52:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:27 smithi131 ceph-mon[29369]: osdmap e622: 8 total, 8 up, 8 in 2024-09-10T18:52:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:27 smithi131 ceph-mon[29369]: pgmap v966: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:27 smithi131 ceph-mon[40885]: osdmap e622: 8 total, 8 up, 8 in 2024-09-10T18:52:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:27 smithi131 ceph-mon[40885]: pgmap v966: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:27 smithi175 ceph-mon[33399]: osdmap e622: 8 total, 8 up, 8 in 2024-09-10T18:52:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:27 smithi175 ceph-mon[33399]: pgmap v966: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:28.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:28 smithi131 ceph-mon[29369]: osdmap e623: 8 total, 8 up, 8 in 2024-09-10T18:52:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:28.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:28 smithi131 ceph-mon[40885]: osdmap e623: 8 total, 8 up, 8 in 2024-09-10T18:52:28.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:28 smithi175 ceph-mon[33399]: osdmap e623: 8 total, 8 up, 8 in 2024-09-10T18:52:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: osdmap e624: 8 total, 8 up, 8 in 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: pgmap v969: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: osdmap e625: 8 total, 8 up, 8 in 2024-09-10T18:52:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-128"}]: dispatch 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: osdmap e624: 8 total, 8 up, 8 in 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: pgmap v969: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: osdmap e625: 8 total, 8 up, 8 in 2024-09-10T18:52:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-128"}]: dispatch 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: osdmap e624: 8 total, 8 up, 8 in 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: pgmap v969: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: osdmap e625: 8 total, 8 up, 8 in 2024-09-10T18:52:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-128"}]: dispatch 2024-09-10T18:52:31.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-128"}]': finished 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[29369]: osdmap e626: 8 total, 8 up, 8 in 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-128", "mode": "writeback"}]: dispatch 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[29369]: pgmap v972: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-128"}]': finished 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[40885]: osdmap e626: 8 total, 8 up, 8 in 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-128", "mode": "writeback"}]: dispatch 2024-09-10T18:52:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:31 smithi131 ceph-mon[40885]: pgmap v972: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-128"}]': finished 2024-09-10T18:52:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:31 smithi175 ceph-mon[33399]: osdmap e626: 8 total, 8 up, 8 in 2024-09-10T18:52:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-128", "mode": "writeback"}]: dispatch 2024-09-10T18:52:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:31 smithi175 ceph-mon[33399]: pgmap v972: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:32 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-128", "mode": "writeback"}]': finished 2024-09-10T18:52:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:32 smithi131 ceph-mon[29369]: osdmap e627: 8 total, 8 up, 8 in 2024-09-10T18:52:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:32 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-128", "mode": "writeback"}]': finished 2024-09-10T18:52:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:32 smithi131 ceph-mon[40885]: osdmap e627: 8 total, 8 up, 8 in 2024-09-10T18:52:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:32 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-128", "mode": "writeback"}]': finished 2024-09-10T18:52:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:32 smithi175 ceph-mon[33399]: osdmap e627: 8 total, 8 up, 8 in 2024-09-10T18:52:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:33 smithi131 ceph-mon[29369]: pgmap v974: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:33 smithi131 ceph-mon[40885]: pgmap v974: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:33.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:33 smithi175 ceph-mon[33399]: pgmap v974: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:52:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:33] "GET /metrics HTTP/1.1" 200 228577 "" "Prometheus/2.43.0" 2024-09-10T18:52:34.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:52:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:52:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:52:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:35.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:35 smithi131 ceph-mon[29369]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T18:52:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:35 smithi131 ceph-mon[40885]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T18:52:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:35 smithi175 ceph-mon[33399]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T18:52:36.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:36.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[29369]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 849 B/s rd, 283 B/s wr, 1 op/s 2024-09-10T18:52:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[29369]: osdmap e628: 8 total, 8 up, 8 in 2024-09-10T18:52:37.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128"}]: dispatch 2024-09-10T18:52:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[40885]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 849 B/s rd, 283 B/s wr, 1 op/s 2024-09-10T18:52:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[40885]: osdmap e628: 8 total, 8 up, 8 in 2024-09-10T18:52:37.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128"}]: dispatch 2024-09-10T18:52:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:37 smithi175 ceph-mon[33399]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 849 B/s rd, 283 B/s wr, 1 op/s 2024-09-10T18:52:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:37 smithi175 ceph-mon[33399]: osdmap e628: 8 total, 8 up, 8 in 2024-09-10T18:52:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128"}]: dispatch 2024-09-10T18:52:38.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:38 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128"}]': finished 2024-09-10T18:52:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:38 smithi131 ceph-mon[29369]: osdmap e629: 8 total, 8 up, 8 in 2024-09-10T18:52:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:38 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128"}]': finished 2024-09-10T18:52:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:38 smithi131 ceph-mon[40885]: osdmap e629: 8 total, 8 up, 8 in 2024-09-10T18:52:38.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:38 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-128"}]': finished 2024-09-10T18:52:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:38 smithi175 ceph-mon[33399]: osdmap e629: 8 total, 8 up, 8 in 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: pgmap v979: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 283 B/s wr, 2 op/s 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: osdmap e630: 8 total, 8 up, 8 in 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: pgmap v979: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 283 B/s wr, 2 op/s 2024-09-10T18:52:39.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:39.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: osdmap e630: 8 total, 8 up, 8 in 2024-09-10T18:52:39.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:39.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:39.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:39.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:39 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:39.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: pgmap v979: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 283 B/s wr, 2 op/s 2024-09-10T18:52:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: osdmap e630: 8 total, 8 up, 8 in 2024-09-10T18:52:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:39 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:40.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:40 smithi131 ceph-mon[29369]: osdmap e631: 8 total, 8 up, 8 in 2024-09-10T18:52:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:40 smithi131 ceph-mon[40885]: osdmap e631: 8 total, 8 up, 8 in 2024-09-10T18:52:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:40.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:40 smithi175 ceph-mon[33399]: osdmap e631: 8 total, 8 up, 8 in 2024-09-10T18:52:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[29369]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[29369]: osdmap e632: 8 total, 8 up, 8 in 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[40885]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[40885]: osdmap e632: 8 total, 8 up, 8 in 2024-09-10T18:52:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:41 smithi175 ceph-mon[33399]: pgmap v982: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:52:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:41 smithi175 ceph-mon[33399]: osdmap e632: 8 total, 8 up, 8 in 2024-09-10T18:52:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[29369]: osdmap e633: 8 total, 8 up, 8 in 2024-09-10T18:52:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-130"}]: dispatch 2024-09-10T18:52:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-130"}]': finished 2024-09-10T18:52:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[29369]: osdmap e634: 8 total, 8 up, 8 in 2024-09-10T18:52:42.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-130", "mode": "writeback"}]: dispatch 2024-09-10T18:52:42.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:42.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[40885]: osdmap e633: 8 total, 8 up, 8 in 2024-09-10T18:52:42.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-130"}]: dispatch 2024-09-10T18:52:42.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-130"}]': finished 2024-09-10T18:52:42.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[40885]: osdmap e634: 8 total, 8 up, 8 in 2024-09-10T18:52:42.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-130", "mode": "writeback"}]: dispatch 2024-09-10T18:52:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:42 smithi175 ceph-mon[33399]: osdmap e633: 8 total, 8 up, 8 in 2024-09-10T18:52:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-130"}]: dispatch 2024-09-10T18:52:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-130"}]': finished 2024-09-10T18:52:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:42 smithi175 ceph-mon[33399]: osdmap e634: 8 total, 8 up, 8 in 2024-09-10T18:52:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-130", "mode": "writeback"}]: dispatch 2024-09-10T18:52:43.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[29369]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:43.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:43.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-130", "mode": "writeback"}]': finished 2024-09-10T18:52:43.804 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[29369]: osdmap e635: 8 total, 8 up, 8 in 2024-09-10T18:52:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[40885]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-130", "mode": "writeback"}]': finished 2024-09-10T18:52:43.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-mon[40885]: osdmap e635: 8 total, 8 up, 8 in 2024-09-10T18:52:43.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:43 smithi175 ceph-mon[33399]: pgmap v985: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:43.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:43 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:43.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-130", "mode": "writeback"}]': finished 2024-09-10T18:52:43.924 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:43 smithi175 ceph-mon[33399]: osdmap e635: 8 total, 8 up, 8 in 2024-09-10T18:52:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:52:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:43] "GET /metrics HTTP/1.1" 200 228577 "" "Prometheus/2.43.0" 2024-09-10T18:52:44.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:52:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:52:44.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:44.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:52:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[29369]: pgmap v988: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[29369]: osdmap e636: 8 total, 8 up, 8 in 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130"}]: dispatch 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[40885]: pgmap v988: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:45.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:45.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[40885]: osdmap e636: 8 total, 8 up, 8 in 2024-09-10T18:52:45.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:45 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130"}]: dispatch 2024-09-10T18:52:45.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:45 smithi175 ceph-mon[33399]: pgmap v988: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:45 smithi175 ceph-mon[33399]: osdmap e636: 8 total, 8 up, 8 in 2024-09-10T18:52:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:45 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130"}]: dispatch 2024-09-10T18:52:46.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:46 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:46.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130"}]': finished 2024-09-10T18:52:46.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:46 smithi131 ceph-mon[29369]: osdmap e637: 8 total, 8 up, 8 in 2024-09-10T18:52:46.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:46 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:46.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130"}]': finished 2024-09-10T18:52:46.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:46 smithi131 ceph-mon[40885]: osdmap e637: 8 total, 8 up, 8 in 2024-09-10T18:52:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:46 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-130"}]': finished 2024-09-10T18:52:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:46 smithi175 ceph-mon[33399]: osdmap e637: 8 total, 8 up, 8 in 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[29369]: pgmap v991: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[29369]: osdmap e638: 8 total, 8 up, 8 in 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[40885]: pgmap v991: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[40885]: osdmap e638: 8 total, 8 up, 8 in 2024-09-10T18:52:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:47.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:47.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:47 smithi175 ceph-mon[33399]: pgmap v991: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:52:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:47 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:47 smithi175 ceph-mon[33399]: osdmap e638: 8 total, 8 up, 8 in 2024-09-10T18:52:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:52:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:52:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:48 smithi175 ceph-mon[33399]: osdmap e639: 8 total, 8 up, 8 in 2024-09-10T18:52:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:49.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:48 smithi131 ceph-mon[29369]: osdmap e639: 8 total, 8 up, 8 in 2024-09-10T18:52:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:48 smithi131 ceph-mon[40885]: osdmap e639: 8 total, 8 up, 8 in 2024-09-10T18:52:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:52:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:49 smithi175 ceph-mon[33399]: pgmap v994: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:49 smithi175 ceph-mon[33399]: osdmap e640: 8 total, 8 up, 8 in 2024-09-10T18:52:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[29369]: pgmap v994: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[29369]: osdmap e640: 8 total, 8 up, 8 in 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[40885]: pgmap v994: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[40885]: osdmap e640: 8 total, 8 up, 8 in 2024-09-10T18:52:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:52:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:50 smithi175 ceph-mon[33399]: osdmap e641: 8 total, 8 up, 8 in 2024-09-10T18:52:50.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:50 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:50 smithi131 ceph-mon[29369]: osdmap e641: 8 total, 8 up, 8 in 2024-09-10T18:52:51.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:50 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:52:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:50 smithi131 ceph-mon[40885]: osdmap e641: 8 total, 8 up, 8 in 2024-09-10T18:52:51.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:50 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: pgmap v997: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: osdmap e642: 8 total, 8 up, 8 in 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-132", "mode": "writeback"}]: dispatch 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-132", "mode": "writeback"}]': finished 2024-09-10T18:52:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:51 smithi175 ceph-mon[33399]: osdmap e643: 8 total, 8 up, 8 in 2024-09-10T18:52:52.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: pgmap v997: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: osdmap e642: 8 total, 8 up, 8 in 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-132", "mode": "writeback"}]: dispatch 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-132", "mode": "writeback"}]': finished 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[29369]: osdmap e643: 8 total, 8 up, 8 in 2024-09-10T18:52:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: pgmap v997: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:52:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: osdmap e642: 8 total, 8 up, 8 in 2024-09-10T18:52:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-132", "mode": "writeback"}]: dispatch 2024-09-10T18:52:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:52:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-132", "mode": "writeback"}]': finished 2024-09-10T18:52:52.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:51 smithi131 ceph-mon[40885]: osdmap e643: 8 total, 8 up, 8 in 2024-09-10T18:52:52.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:52 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:53.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:52 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:52 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:52:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:53 smithi175 ceph-mon[33399]: pgmap v1000: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:53 smithi175 ceph-mon[33399]: osdmap e644: 8 total, 8 up, 8 in 2024-09-10T18:52:54.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:53 smithi131 ceph-mon[29369]: pgmap v1000: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:53 smithi131 ceph-mon[29369]: osdmap e644: 8 total, 8 up, 8 in 2024-09-10T18:52:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:52:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:53] "GET /metrics HTTP/1.1" 200 226141 "" "Prometheus/2.43.0" 2024-09-10T18:52:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:53 smithi131 ceph-mon[40885]: pgmap v1000: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:52:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:53 smithi131 ceph-mon[40885]: osdmap e644: 8 total, 8 up, 8 in 2024-09-10T18:52:54.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:52:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:52:54.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:54 smithi175 ceph-mon[33399]: osdmap e645: 8 total, 8 up, 8 in 2024-09-10T18:52:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:55.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:54 smithi131 ceph-mon[29369]: osdmap e645: 8 total, 8 up, 8 in 2024-09-10T18:52:55.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:54 smithi131 ceph-mon[40885]: osdmap e645: 8 total, 8 up, 8 in 2024-09-10T18:52:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:52:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:52:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:52:55.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:55 smithi175 ceph-mon[33399]: pgmap v1003: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:55 smithi175 ceph-mon[33399]: osdmap e646: 8 total, 8 up, 8 in 2024-09-10T18:52:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[29369]: pgmap v1003: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[29369]: osdmap e646: 8 total, 8 up, 8 in 2024-09-10T18:52:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:52:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[40885]: pgmap v1003: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[40885]: osdmap e646: 8 total, 8 up, 8 in 2024-09-10T18:52:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:56.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:56 smithi175 ceph-mon[33399]: osdmap e647: 8 total, 8 up, 8 in 2024-09-10T18:52:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:56 smithi175 ceph-mon[33399]: osdmap e648: 8 total, 8 up, 8 in 2024-09-10T18:52:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:56 smithi131 ceph-mon[40885]: osdmap e647: 8 total, 8 up, 8 in 2024-09-10T18:52:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:56 smithi131 ceph-mon[40885]: osdmap e648: 8 total, 8 up, 8 in 2024-09-10T18:52:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:56 smithi131 ceph-mon[29369]: osdmap e647: 8 total, 8 up, 8 in 2024-09-10T18:52:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:56 smithi131 ceph-mon[29369]: osdmap e648: 8 total, 8 up, 8 in 2024-09-10T18:52:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:57 smithi175 ceph-mon[33399]: pgmap v1006: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:57 smithi131 ceph-mon[40885]: pgmap v1006: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:57 smithi131 ceph-mon[29369]: pgmap v1006: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:52:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:52:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:58 smithi175 ceph-mon[33399]: osdmap e649: 8 total, 8 up, 8 in 2024-09-10T18:52:58.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:59.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:58 smithi131 ceph-mon[29369]: osdmap e649: 8 total, 8 up, 8 in 2024-09-10T18:52:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:52:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:58 smithi131 ceph-mon[40885]: osdmap e649: 8 total, 8 up, 8 in 2024-09-10T18:52:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132"}]: dispatch 2024-09-10T18:52:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:59 smithi175 ceph-mon[33399]: pgmap v1009: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:52:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:59 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:52:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:52:59.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:52:59 smithi175 ceph-mon[33399]: osdmap e650: 8 total, 8 up, 8 in 2024-09-10T18:53:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[29369]: pgmap v1009: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:53:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:53:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[29369]: osdmap e650: 8 total, 8 up, 8 in 2024-09-10T18:53:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[40885]: pgmap v1009: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T18:53:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-132"}]': finished 2024-09-10T18:53:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:52:59 smithi131 ceph-mon[40885]: osdmap e650: 8 total, 8 up, 8 in 2024-09-10T18:53:00.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:00 smithi175 ceph-mon[33399]: osdmap e651: 8 total, 8 up, 8 in 2024-09-10T18:53:01.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:00 smithi131 ceph-mon[29369]: osdmap e651: 8 total, 8 up, 8 in 2024-09-10T18:53:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:00 smithi131 ceph-mon[40885]: osdmap e651: 8 total, 8 up, 8 in 2024-09-10T18:53:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:01 smithi131 ceph-mon[29369]: pgmap v1012: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:53:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:01 smithi131 ceph-mon[29369]: osdmap e652: 8 total, 8 up, 8 in 2024-09-10T18:53:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:01 smithi131 ceph-mon[40885]: pgmap v1012: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:53:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:01 smithi131 ceph-mon[40885]: osdmap e652: 8 total, 8 up, 8 in 2024-09-10T18:53:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:01 smithi175 ceph-mon[33399]: pgmap v1012: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:53:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:01 smithi175 ceph-mon[33399]: osdmap e652: 8 total, 8 up, 8 in 2024-09-10T18:53:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: osdmap e653: 8 total, 8 up, 8 in 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: osdmap e654: 8 total, 8 up, 8 in 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-134"}]: dispatch 2024-09-10T18:53:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: osdmap e653: 8 total, 8 up, 8 in 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: osdmap e654: 8 total, 8 up, 8 in 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-134"}]: dispatch 2024-09-10T18:53:03.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:03.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: osdmap e653: 8 total, 8 up, 8 in 2024-09-10T18:53:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: osdmap e654: 8 total, 8 up, 8 in 2024-09-10T18:53:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-134"}]: dispatch 2024-09-10T18:53:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:03.946 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:03 smithi175 ceph-mon[33399]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:03.946 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-134"}]': finished 2024-09-10T18:53:03.946 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:03 smithi175 ceph-mon[33399]: osdmap e655: 8 total, 8 up, 8 in 2024-09-10T18:53:03.946 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-134", "mode": "writeback"}]: dispatch 2024-09-10T18:53:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:03] "GET /metrics HTTP/1.1" 200 228599 "" "Prometheus/2.43.0" 2024-09-10T18:53:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[29369]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-134"}]': finished 2024-09-10T18:53:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[29369]: osdmap e655: 8 total, 8 up, 8 in 2024-09-10T18:53:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-134", "mode": "writeback"}]: dispatch 2024-09-10T18:53:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[40885]: pgmap v1016: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-134"}]': finished 2024-09-10T18:53:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[40885]: osdmap e655: 8 total, 8 up, 8 in 2024-09-10T18:53:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-134", "mode": "writeback"}]: dispatch 2024-09-10T18:53:04.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:53:03 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-134", "mode": "writeback"}]': finished 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[29369]: osdmap e656: 8 total, 8 up, 8 in 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-134", "mode": "writeback"}]': finished 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[40885]: osdmap e656: 8 total, 8 up, 8 in 2024-09-10T18:53:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:05.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:04 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:05.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-134", "mode": "writeback"}]': finished 2024-09-10T18:53:05.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:04 smithi175 ceph-mon[33399]: osdmap e656: 8 total, 8 up, 8 in 2024-09-10T18:53:05.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:53:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[29369]: pgmap v1019: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[29369]: osdmap e657: 8 total, 8 up, 8 in 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134"}]: dispatch 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[40885]: pgmap v1019: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:53:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[40885]: osdmap e657: 8 total, 8 up, 8 in 2024-09-10T18:53:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134"}]: dispatch 2024-09-10T18:53:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:05 smithi175 ceph-mon[33399]: pgmap v1019: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:53:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:05 smithi175 ceph-mon[33399]: osdmap e657: 8 total, 8 up, 8 in 2024-09-10T18:53:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134"}]: dispatch 2024-09-10T18:53:07.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:06 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134"}]': finished 2024-09-10T18:53:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:06 smithi131 ceph-mon[29369]: osdmap e658: 8 total, 8 up, 8 in 2024-09-10T18:53:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:06 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134"}]': finished 2024-09-10T18:53:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:06 smithi131 ceph-mon[40885]: osdmap e658: 8 total, 8 up, 8 in 2024-09-10T18:53:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:06 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-134"}]': finished 2024-09-10T18:53:07.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:06 smithi175 ceph-mon[33399]: osdmap e658: 8 total, 8 up, 8 in 2024-09-10T18:53:08.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:07 smithi131 ceph-mon[29369]: pgmap v1022: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:07 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:07 smithi131 ceph-mon[29369]: osdmap e659: 8 total, 8 up, 8 in 2024-09-10T18:53:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:07 smithi131 ceph-mon[40885]: pgmap v1022: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:07 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:07 smithi131 ceph-mon[40885]: osdmap e659: 8 total, 8 up, 8 in 2024-09-10T18:53:08.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:07 smithi175 ceph-mon[33399]: pgmap v1022: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:07 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:07 smithi175 ceph-mon[33399]: osdmap e659: 8 total, 8 up, 8 in 2024-09-10T18:53:09.576 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:09 smithi131 ceph-mon[40885]: osdmap e660: 8 total, 8 up, 8 in 2024-09-10T18:53:09.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:09.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:09 smithi131 ceph-mon[40885]: pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:53:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:09 smithi131 ceph-mon[29369]: osdmap e660: 8 total, 8 up, 8 in 2024-09-10T18:53:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:09.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:09 smithi131 ceph-mon[29369]: pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:53:09.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:09 smithi175 ceph-mon[33399]: osdmap e660: 8 total, 8 up, 8 in 2024-09-10T18:53:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:09.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:09 smithi175 ceph-mon[33399]: pgmap v1025: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:53:10.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:10 smithi131 ceph-mon[29369]: osdmap e661: 8 total, 8 up, 8 in 2024-09-10T18:53:10.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:10 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:10.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:10.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:10 smithi131 ceph-mon[40885]: osdmap e661: 8 total, 8 up, 8 in 2024-09-10T18:53:10.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:10 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:10.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:10.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:10 smithi175 ceph-mon[33399]: osdmap e661: 8 total, 8 up, 8 in 2024-09-10T18:53:10.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:10 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[29369]: osdmap e662: 8 total, 8 up, 8 in 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-136"}]: dispatch 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[29369]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[40885]: osdmap e662: 8 total, 8 up, 8 in 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-136"}]: dispatch 2024-09-10T18:53:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:11 smithi131 ceph-mon[40885]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:53:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:11 smithi175 ceph-mon[33399]: osdmap e662: 8 total, 8 up, 8 in 2024-09-10T18:53:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-136"}]: dispatch 2024-09-10T18:53:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:11 smithi175 ceph-mon[33399]: pgmap v1028: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-136"}]': finished 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[29369]: osdmap e663: 8 total, 8 up, 8 in 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-136", "mode": "writeback"}]: dispatch 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-136", "mode": "writeback"}]': finished 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[29369]: osdmap e664: 8 total, 8 up, 8 in 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-136"}]': finished 2024-09-10T18:53:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[40885]: osdmap e663: 8 total, 8 up, 8 in 2024-09-10T18:53:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-136", "mode": "writeback"}]: dispatch 2024-09-10T18:53:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-136", "mode": "writeback"}]': finished 2024-09-10T18:53:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:12 smithi131 ceph-mon[40885]: osdmap e664: 8 total, 8 up, 8 in 2024-09-10T18:53:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-136"}]': finished 2024-09-10T18:53:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:12 smithi175 ceph-mon[33399]: osdmap e663: 8 total, 8 up, 8 in 2024-09-10T18:53:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-136", "mode": "writeback"}]: dispatch 2024-09-10T18:53:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:12 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-136", "mode": "writeback"}]': finished 2024-09-10T18:53:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:12 smithi175 ceph-mon[33399]: osdmap e664: 8 total, 8 up, 8 in 2024-09-10T18:53:13.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:13 smithi131 ceph-mon[29369]: pgmap v1031: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:13 smithi131 ceph-mon[40885]: pgmap v1031: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:13 smithi175 ceph-mon[33399]: pgmap v1031: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:14.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:53:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:13] "GET /metrics HTTP/1.1" 200 228599 "" "Prometheus/2.43.0" 2024-09-10T18:53:14.229 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:53:13 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:53:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:14 smithi131 ceph-mon[29369]: osdmap e665: 8 total, 8 up, 8 in 2024-09-10T18:53:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136"}]: dispatch 2024-09-10T18:53:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:14 smithi131 ceph-mon[40885]: osdmap e665: 8 total, 8 up, 8 in 2024-09-10T18:53:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136"}]: dispatch 2024-09-10T18:53:14.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:14 smithi175 ceph-mon[33399]: osdmap e665: 8 total, 8 up, 8 in 2024-09-10T18:53:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136"}]: dispatch 2024-09-10T18:53:15.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:53:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:15 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136"}]': finished 2024-09-10T18:53:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:15 smithi175 ceph-mon[33399]: osdmap e666: 8 total, 8 up, 8 in 2024-09-10T18:53:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:15 smithi175 ceph-mon[33399]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136"}]': finished 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[29369]: osdmap e666: 8 total, 8 up, 8 in 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[29369]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-136"}]': finished 2024-09-10T18:53:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[40885]: osdmap e666: 8 total, 8 up, 8 in 2024-09-10T18:53:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:15 smithi131 ceph-mon[40885]: pgmap v1034: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-09-10T18:53:16.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:16 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:16 smithi131 ceph-mon[29369]: osdmap e667: 8 total, 8 up, 8 in 2024-09-10T18:53:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:16 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:16 smithi131 ceph-mon[40885]: osdmap e667: 8 total, 8 up, 8 in 2024-09-10T18:53:16.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:16 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:16 smithi175 ceph-mon[33399]: osdmap e667: 8 total, 8 up, 8 in 2024-09-10T18:53:17.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:17 smithi131 ceph-mon[29369]: osdmap e668: 8 total, 8 up, 8 in 2024-09-10T18:53:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:17 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:17 smithi131 ceph-mon[29369]: pgmap v1037: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:53:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:17 smithi131 ceph-mon[40885]: osdmap e668: 8 total, 8 up, 8 in 2024-09-10T18:53:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:17 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:17 smithi131 ceph-mon[40885]: pgmap v1037: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:53:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:17 smithi175 ceph-mon[33399]: osdmap e668: 8 total, 8 up, 8 in 2024-09-10T18:53:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:17 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:17 smithi175 ceph-mon[33399]: pgmap v1037: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-09-10T18:53:18.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[29369]: osdmap e669: 8 total, 8 up, 8 in 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[40885]: osdmap e669: 8 total, 8 up, 8 in 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:18.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:18 smithi175 ceph-mon[33399]: osdmap e669: 8 total, 8 up, 8 in 2024-09-10T18:53:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:19.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: osdmap e670: 8 total, 8 up, 8 in 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: pgmap v1040: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: osdmap e671: 8 total, 8 up, 8 in 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: osdmap e670: 8 total, 8 up, 8 in 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:53:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: pgmap v1040: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:53:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: osdmap e671: 8 total, 8 up, 8 in 2024-09-10T18:53:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: osdmap e670: 8 total, 8 up, 8 in 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: pgmap v1040: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: osdmap e671: 8 total, 8 up, 8 in 2024-09-10T18:53:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[29369]: osdmap e672: 8 total, 8 up, 8 in 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[29369]: pgmap v1043: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[40885]: osdmap e672: 8 total, 8 up, 8 in 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:53:21.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:21 smithi131 ceph-mon[40885]: pgmap v1043: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:53:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:21 smithi175 ceph-mon[33399]: osdmap e672: 8 total, 8 up, 8 in 2024-09-10T18:53:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:53:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:21 smithi175 ceph-mon[33399]: pgmap v1043: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:22.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:53:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:22 smithi131 ceph-mon[29369]: osdmap e673: 8 total, 8 up, 8 in 2024-09-10T18:53:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:22 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:53:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:22 smithi131 ceph-mon[40885]: osdmap e673: 8 total, 8 up, 8 in 2024-09-10T18:53:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:22 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:22.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:53:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:22 smithi175 ceph-mon[33399]: osdmap e673: 8 total, 8 up, 8 in 2024-09-10T18:53:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:22 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138"}]: dispatch 2024-09-10T18:53:23.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-mon[29369]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138"}]: dispatch 2024-09-10T18:53:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-mon[40885]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:23.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:23.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138"}]: dispatch 2024-09-10T18:53:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:23 smithi175 ceph-mon[33399]: pgmap v1045: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:24.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:53:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:23] "GET /metrics HTTP/1.1" 200 228598 "" "Prometheus/2.43.0" 2024-09-10T18:53:24.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:53:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:53:24.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138"}]': finished 2024-09-10T18:53:24.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:24 smithi131 ceph-mon[40885]: osdmap e674: 8 total, 8 up, 8 in 2024-09-10T18:53:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138"}]': finished 2024-09-10T18:53:24.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:24 smithi131 ceph-mon[29369]: osdmap e674: 8 total, 8 up, 8 in 2024-09-10T18:53:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-138"}]': finished 2024-09-10T18:53:24.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:24 smithi175 ceph-mon[33399]: osdmap e674: 8 total, 8 up, 8 in 2024-09-10T18:53:25.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:53:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[29369]: osdmap e675: 8 total, 8 up, 8 in 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[29369]: pgmap v1048: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[29369]: osdmap e676: 8 total, 8 up, 8 in 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[40885]: osdmap e675: 8 total, 8 up, 8 in 2024-09-10T18:53:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[40885]: pgmap v1048: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[40885]: osdmap e676: 8 total, 8 up, 8 in 2024-09-10T18:53:25.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:25.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:25 smithi175 ceph-mon[33399]: osdmap e675: 8 total, 8 up, 8 in 2024-09-10T18:53:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:25 smithi175 ceph-mon[33399]: pgmap v1048: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:25 smithi175 ceph-mon[33399]: osdmap e676: 8 total, 8 up, 8 in 2024-09-10T18:53:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[40885]: osdmap e677: 8 total, 8 up, 8 in 2024-09-10T18:53:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[40885]: pgmap v1051: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:27.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[29369]: osdmap e677: 8 total, 8 up, 8 in 2024-09-10T18:53:27.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:27.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:27 smithi131 ceph-mon[29369]: pgmap v1051: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:27.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:27.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:27 smithi175 ceph-mon[33399]: osdmap e677: 8 total, 8 up, 8 in 2024-09-10T18:53:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:27 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:27.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:27 smithi175 ceph-mon[33399]: pgmap v1051: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:28 smithi131 ceph-mon[29369]: osdmap e678: 8 total, 8 up, 8 in 2024-09-10T18:53:28.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:28 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:53:28.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:28.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:28 smithi131 ceph-mon[40885]: osdmap e678: 8 total, 8 up, 8 in 2024-09-10T18:53:28.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:28 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:53:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:28 smithi175 ceph-mon[33399]: osdmap e678: 8 total, 8 up, 8 in 2024-09-10T18:53:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:28 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: osdmap e679: 8 total, 8 up, 8 in 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: pgmap v1054: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: osdmap e680: 8 total, 8 up, 8 in 2024-09-10T18:53:29.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: osdmap e679: 8 total, 8 up, 8 in 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: pgmap v1054: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: osdmap e680: 8 total, 8 up, 8 in 2024-09-10T18:53:29.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:29 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: osdmap e679: 8 total, 8 up, 8 in 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: pgmap v1054: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: osdmap e680: 8 total, 8 up, 8 in 2024-09-10T18:53:29.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:29 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[29369]: osdmap e681: 8 total, 8 up, 8 in 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[29369]: pgmap v1057: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[40885]: osdmap e681: 8 total, 8 up, 8 in 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:53:31.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:31 smithi131 ceph-mon[40885]: pgmap v1057: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:53:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:31 smithi175 ceph-mon[33399]: osdmap e681: 8 total, 8 up, 8 in 2024-09-10T18:53:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:53:31.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:31 smithi175 ceph-mon[33399]: pgmap v1057: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:32.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:53:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:32 smithi131 ceph-mon[29369]: osdmap e682: 8 total, 8 up, 8 in 2024-09-10T18:53:32.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:53:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:32 smithi131 ceph-mon[40885]: osdmap e682: 8 total, 8 up, 8 in 2024-09-10T18:53:32.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:53:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:32 smithi175 ceph-mon[33399]: osdmap e682: 8 total, 8 up, 8 in 2024-09-10T18:53:32.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:33.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:33 smithi131 ceph-mon[29369]: pgmap v1059: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:33 smithi131 ceph-mon[40885]: pgmap v1059: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:33.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:33 smithi175 ceph-mon[33399]: pgmap v1059: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:34.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:53:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:33] "GET /metrics HTTP/1.1" 200 228588 "" "Prometheus/2.43.0" 2024-09-10T18:53:34.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:53:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:53:35.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:53:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:35.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:35 smithi131 ceph-mon[29369]: pgmap v1060: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-10T18:53:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:35 smithi131 ceph-mon[40885]: pgmap v1060: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-10T18:53:35.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:35 smithi175 ceph-mon[33399]: pgmap v1060: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-09-10T18:53:37.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:37 smithi131 ceph-mon[29369]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 717 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-10T18:53:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:37 smithi131 ceph-mon[40885]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 717 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-10T18:53:37.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:37 smithi175 ceph-mon[33399]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 717 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-10T18:53:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:39 smithi131 ceph-mon[29369]: pgmap v1062: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-09-10T18:53:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:39 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:39 smithi131 ceph-mon[40885]: pgmap v1062: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-09-10T18:53:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:39 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:39 smithi175 ceph-mon[33399]: pgmap v1062: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.1 KiB/s wr, 2 op/s 2024-09-10T18:53:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:39 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:53:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:53:40.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:53:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:41 smithi175 ceph-mon[33399]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-09-10T18:53:42.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:41 smithi131 ceph-mon[29369]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-09-10T18:53:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:41 smithi131 ceph-mon[40885]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-09-10T18:53:43.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:43 smithi175 ceph-mon[33399]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 919 B/s rd, 5.2 KiB/s wr, 1 op/s 2024-09-10T18:53:43.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:43.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140"}]: dispatch 2024-09-10T18:53:44.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-mon[29369]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 919 B/s rd, 5.2 KiB/s wr, 1 op/s 2024-09-10T18:53:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140"}]: dispatch 2024-09-10T18:53:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:43] "GET /metrics HTTP/1.1" 200 228588 "" "Prometheus/2.43.0" 2024-09-10T18:53:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-mon[40885]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 919 B/s rd, 5.2 KiB/s wr, 1 op/s 2024-09-10T18:53:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140"}]: dispatch 2024-09-10T18:53:44.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:53:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:53:44.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140"}]': finished 2024-09-10T18:53:44.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:44 smithi175 ceph-mon[33399]: osdmap e683: 8 total, 8 up, 8 in 2024-09-10T18:53:45.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140"}]': finished 2024-09-10T18:53:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:44 smithi131 ceph-mon[29369]: osdmap e683: 8 total, 8 up, 8 in 2024-09-10T18:53:45.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-140"}]': finished 2024-09-10T18:53:45.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:44 smithi131 ceph-mon[40885]: osdmap e683: 8 total, 8 up, 8 in 2024-09-10T18:53:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:53:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:45 smithi175 ceph-mon[33399]: pgmap v1066: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:53:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:45 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:45 smithi175 ceph-mon[33399]: osdmap e684: 8 total, 8 up, 8 in 2024-09-10T18:53:46.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[29369]: pgmap v1066: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[29369]: osdmap e684: 8 total, 8 up, 8 in 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[40885]: pgmap v1066: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:45 smithi131 ceph-mon[40885]: osdmap e684: 8 total, 8 up, 8 in 2024-09-10T18:53:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:46 smithi175 ceph-mon[33399]: osdmap e685: 8 total, 8 up, 8 in 2024-09-10T18:53:46.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:47.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:46 smithi131 ceph-mon[29369]: osdmap e685: 8 total, 8 up, 8 in 2024-09-10T18:53:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:46 smithi131 ceph-mon[40885]: osdmap e685: 8 total, 8 up, 8 in 2024-09-10T18:53:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:53:47.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:47 smithi175 ceph-mon[33399]: pgmap v1069: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:47.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:47.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:47 smithi175 ceph-mon[33399]: osdmap e686: 8 total, 8 up, 8 in 2024-09-10T18:53:47.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:47.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:48.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[29369]: pgmap v1069: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:48.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[29369]: osdmap e686: 8 total, 8 up, 8 in 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[40885]: pgmap v1069: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[40885]: osdmap e686: 8 total, 8 up, 8 in 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:53:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:53:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:48 smithi175 ceph-mon[33399]: osdmap e687: 8 total, 8 up, 8 in 2024-09-10T18:53:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:48 smithi131 ceph-mon[29369]: osdmap e687: 8 total, 8 up, 8 in 2024-09-10T18:53:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:53:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:48 smithi131 ceph-mon[40885]: osdmap e687: 8 total, 8 up, 8 in 2024-09-10T18:53:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-142"}]': finished 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: osdmap e688: 8 total, 8 up, 8 in 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-142", "mode": "writeback"}]: dispatch 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-142", "mode": "writeback"}]': finished 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: osdmap e689: 8 total, 8 up, 8 in 2024-09-10T18:53:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-142"}]': finished 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: osdmap e688: 8 total, 8 up, 8 in 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-142", "mode": "writeback"}]: dispatch 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-142", "mode": "writeback"}]': finished 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: osdmap e689: 8 total, 8 up, 8 in 2024-09-10T18:53:50.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: pgmap v1072: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-142"}]': finished 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: osdmap e688: 8 total, 8 up, 8 in 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-142", "mode": "writeback"}]: dispatch 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-142", "mode": "writeback"}]': finished 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: osdmap e689: 8 total, 8 up, 8 in 2024-09-10T18:53:50.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:53:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:51 smithi175 ceph-mon[33399]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:53:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:53:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:51 smithi175 ceph-mon[33399]: osdmap e690: 8 total, 8 up, 8 in 2024-09-10T18:53:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:53:52.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[29369]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[29369]: osdmap e690: 8 total, 8 up, 8 in 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[40885]: pgmap v1075: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[40885]: osdmap e690: 8 total, 8 up, 8 in 2024-09-10T18:53:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: osdmap e691: 8 total, 8 up, 8 in 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:53:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:52 smithi175 ceph-mon[33399]: osdmap e692: 8 total, 8 up, 8 in 2024-09-10T18:53:53.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:53:53.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: osdmap e691: 8 total, 8 up, 8 in 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[29369]: osdmap e692: 8 total, 8 up, 8 in 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: osdmap e691: 8 total, 8 up, 8 in 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:53:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:52 smithi131 ceph-mon[40885]: osdmap e692: 8 total, 8 up, 8 in 2024-09-10T18:53:53.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:53 smithi175 ceph-mon[33399]: pgmap v1078: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:53:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:53:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:53 smithi175 ceph-mon[33399]: osdmap e693: 8 total, 8 up, 8 in 2024-09-10T18:53:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:53 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:53:54.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:53] "GET /metrics HTTP/1.1" 200 228533 "" "Prometheus/2.43.0" 2024-09-10T18:53:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[29369]: pgmap v1078: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:53:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:53:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[29369]: osdmap e693: 8 total, 8 up, 8 in 2024-09-10T18:53:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:53:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[40885]: pgmap v1078: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:54.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:53:54.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:53:54.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[40885]: osdmap e693: 8 total, 8 up, 8 in 2024-09-10T18:53:54.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:53 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T18:53:54.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:53:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:53:55.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:53:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:53:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:53:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:55 smithi175 ceph-mon[33399]: pgmap v1081: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:53:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:55 smithi175 ceph-mon[33399]: osdmap e694: 8 total, 8 up, 8 in 2024-09-10T18:53:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:53:56.076 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[40885]: pgmap v1081: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[40885]: osdmap e694: 8 total, 8 up, 8 in 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[29369]: pgmap v1081: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[29369]: osdmap e694: 8 total, 8 up, 8 in 2024-09-10T18:53:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T18:53:56.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:53:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:56 smithi175 ceph-mon[33399]: osdmap e695: 8 total, 8 up, 8 in 2024-09-10T18:53:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:57.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:53:57.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:56 smithi131 ceph-mon[29369]: osdmap e695: 8 total, 8 up, 8 in 2024-09-10T18:53:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T18:53:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:56 smithi131 ceph-mon[40885]: osdmap e695: 8 total, 8 up, 8 in 2024-09-10T18:53:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[29369]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[29369]: osdmap e696: 8 total, 8 up, 8 in 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[40885]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[40885]: osdmap e696: 8 total, 8 up, 8 in 2024-09-10T18:53:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:58.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:57 smithi175 ceph-mon[33399]: pgmap v1084: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:53:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:57 smithi175 ceph-mon[33399]: osdmap e696: 8 total, 8 up, 8 in 2024-09-10T18:53:58.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:59.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]': finished 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[29369]: osdmap e697: 8 total, 8 up, 8 in 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]': finished 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[40885]: osdmap e697: 8 total, 8 up, 8 in 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:53:59.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]': finished 2024-09-10T18:53:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:58 smithi175 ceph-mon[33399]: osdmap e697: 8 total, 8 up, 8 in 2024-09-10T18:53:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:53:59.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-142"}]: dispatch 2024-09-10T18:54:00.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:59 smithi131 ceph-mon[29369]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:54:00.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:59 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:53:59 smithi131 ceph-mon[29369]: osdmap e698: 8 total, 8 up, 8 in 2024-09-10T18:54:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:59 smithi131 ceph-mon[40885]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:54:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:59 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:53:59 smithi131 ceph-mon[40885]: osdmap e698: 8 total, 8 up, 8 in 2024-09-10T18:54:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:59 smithi175 ceph-mon[33399]: pgmap v1087: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T18:54:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:59 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:53:59 smithi175 ceph-mon[33399]: osdmap e698: 8 total, 8 up, 8 in 2024-09-10T18:54:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:00 smithi131 ceph-mon[29369]: osdmap e699: 8 total, 8 up, 8 in 2024-09-10T18:54:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:00 smithi131 ceph-mon[40885]: osdmap e699: 8 total, 8 up, 8 in 2024-09-10T18:54:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:01.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:00 smithi175 ceph-mon[33399]: osdmap e699: 8 total, 8 up, 8 in 2024-09-10T18:54:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[29369]: pgmap v1090: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[29369]: osdmap e700: 8 total, 8 up, 8 in 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[40885]: pgmap v1090: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[40885]: osdmap e700: 8 total, 8 up, 8 in 2024-09-10T18:54:01.978 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:01 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:02.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:01 smithi175 ceph-mon[33399]: pgmap v1090: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:54:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:01 smithi175 ceph-mon[33399]: osdmap e700: 8 total, 8 up, 8 in 2024-09-10T18:54:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:01 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:03.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[29369]: osdmap e701: 8 total, 8 up, 8 in 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[40885]: osdmap e701: 8 total, 8 up, 8 in 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:03.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:02 smithi175 ceph-mon[33399]: osdmap e701: 8 total, 8 up, 8 in 2024-09-10T18:54:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:02 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:04.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:03 smithi175 ceph-mon[33399]: pgmap v1093: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:04.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-144"}]': finished 2024-09-10T18:54:04.001 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:03 smithi175 ceph-mon[33399]: osdmap e702: 8 total, 8 up, 8 in 2024-09-10T18:54:04.002 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:03 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-144", "mode": "readproxy"}]: dispatch 2024-09-10T18:54:04.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:03] "GET /metrics HTTP/1.1" 200 228585 "" "Prometheus/2.43.0" 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[29369]: pgmap v1093: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-144"}]': finished 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[29369]: osdmap e702: 8 total, 8 up, 8 in 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-144", "mode": "readproxy"}]: dispatch 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[40885]: pgmap v1093: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-144"}]': finished 2024-09-10T18:54:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[40885]: osdmap e702: 8 total, 8 up, 8 in 2024-09-10T18:54:04.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:03 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-144", "mode": "readproxy"}]: dispatch 2024-09-10T18:54:04.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:54:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:54:05.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:04 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:54:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-144", "mode": "readproxy"}]': finished 2024-09-10T18:54:05.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:04 smithi131 ceph-mon[29369]: osdmap e703: 8 total, 8 up, 8 in 2024-09-10T18:54:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:04 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:54:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-144", "mode": "readproxy"}]': finished 2024-09-10T18:54:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:04 smithi131 ceph-mon[40885]: osdmap e703: 8 total, 8 up, 8 in 2024-09-10T18:54:05.077 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:04 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:54:05.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-144", "mode": "readproxy"}]': finished 2024-09-10T18:54:05.078 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:04 smithi175 ceph-mon[33399]: osdmap e703: 8 total, 8 up, 8 in 2024-09-10T18:54:05.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:54:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:06.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:05 smithi131 ceph-mon[29369]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:54:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:05 smithi131 ceph-mon[40885]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:54:06.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:05 smithi175 ceph-mon[33399]: pgmap v1096: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:54:08.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:07 smithi131 ceph-mon[29369]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 889 B/s rd, 177 B/s wr, 1 op/s 2024-09-10T18:54:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:07 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:07 smithi131 ceph-mon[40885]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 889 B/s rd, 177 B/s wr, 1 op/s 2024-09-10T18:54:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:07 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:07 smithi175 ceph-mon[33399]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 889 B/s rd, 177 B/s wr, 1 op/s 2024-09-10T18:54:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:07 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:09 smithi131 ceph-mon[29369]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-09-10T18:54:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:09 smithi131 ceph-mon[40885]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-09-10T18:54:10.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:09 smithi175 ceph-mon[33399]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-09-10T18:54:12.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:11 smithi131 ceph-mon[40885]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-10T18:54:12.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:11 smithi131 ceph-mon[29369]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-10T18:54:12.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:11 smithi175 ceph-mon[33399]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-10T18:54:13.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:13 smithi131 ceph-mon[29369]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 105 B/s wr, 1 op/s 2024-09-10T18:54:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:13 smithi131 ceph-mon[40885]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 105 B/s wr, 1 op/s 2024-09-10T18:54:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:13 smithi175 ceph-mon[33399]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 105 B/s wr, 1 op/s 2024-09-10T18:54:14.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:54:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:13] "GET /metrics HTTP/1.1" 200 228585 "" "Prometheus/2.43.0" 2024-09-10T18:54:14.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:14.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:54:14 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:54:14.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:15.328 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:15.328 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: osdmap e704: 8 total, 8 up, 8 in 2024-09-10T18:54:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: pgmap v1102: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:54:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:54:15.329 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]': finished 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: osdmap e704: 8 total, 8 up, 8 in 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: pgmap v1102: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]': finished 2024-09-10T18:54:15.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: osdmap e704: 8 total, 8 up, 8 in 2024-09-10T18:54:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: pgmap v1102: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:54:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:54:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]': finished 2024-09-10T18:54:16.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:16 smithi131 ceph-mon[29369]: osdmap e705: 8 total, 8 up, 8 in 2024-09-10T18:54:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:16 smithi131 ceph-mon[40885]: osdmap e705: 8 total, 8 up, 8 in 2024-09-10T18:54:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:16 smithi175 ceph-mon[33399]: osdmap e705: 8 total, 8 up, 8 in 2024-09-10T18:54:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-144"}]: dispatch 2024-09-10T18:54:17.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:17 smithi131 ceph-mon[29369]: osdmap e706: 8 total, 8 up, 8 in 2024-09-10T18:54:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:17 smithi131 ceph-mon[29369]: pgmap v1105: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:17 smithi131 ceph-mon[40885]: osdmap e706: 8 total, 8 up, 8 in 2024-09-10T18:54:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:17 smithi131 ceph-mon[40885]: pgmap v1105: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:17.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:17 smithi175 ceph-mon[33399]: osdmap e706: 8 total, 8 up, 8 in 2024-09-10T18:54:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:17 smithi175 ceph-mon[33399]: pgmap v1105: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:18 smithi131 ceph-mon[29369]: osdmap e707: 8 total, 8 up, 8 in 2024-09-10T18:54:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:18 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:18.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:18 smithi131 ceph-mon[40885]: osdmap e707: 8 total, 8 up, 8 in 2024-09-10T18:54:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:18 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:18.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:18 smithi175 ceph-mon[33399]: osdmap e707: 8 total, 8 up, 8 in 2024-09-10T18:54:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:18 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:18.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[29369]: osdmap e708: 8 total, 8 up, 8 in 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[29369]: pgmap v1108: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[40885]: osdmap e708: 8 total, 8 up, 8 in 2024-09-10T18:54:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:19.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:19 smithi131 ceph-mon[40885]: pgmap v1108: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:19 smithi175 ceph-mon[33399]: osdmap e708: 8 total, 8 up, 8 in 2024-09-10T18:54:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:19 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:19.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:19 smithi175 ceph-mon[33399]: pgmap v1108: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:20 smithi131 ceph-mon[29369]: osdmap e709: 8 total, 8 up, 8 in 2024-09-10T18:54:20.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:20 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:20.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:20.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:20 smithi131 ceph-mon[40885]: osdmap e709: 8 total, 8 up, 8 in 2024-09-10T18:54:20.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:20 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:20 smithi175 ceph-mon[33399]: osdmap e709: 8 total, 8 up, 8 in 2024-09-10T18:54:20.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:20 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:21.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-146"}]': finished 2024-09-10T18:54:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: osdmap e710: 8 total, 8 up, 8 in 2024-09-10T18:54:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-146", "mode": "writeback"}]: dispatch 2024-09-10T18:54:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: pgmap v1111: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:54:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-146", "mode": "writeback"}]': finished 2024-09-10T18:54:21.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[29369]: osdmap e711: 8 total, 8 up, 8 in 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-146"}]': finished 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: osdmap e710: 8 total, 8 up, 8 in 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-146", "mode": "writeback"}]: dispatch 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: pgmap v1111: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-146", "mode": "writeback"}]': finished 2024-09-10T18:54:21.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:21 smithi131 ceph-mon[40885]: osdmap e711: 8 total, 8 up, 8 in 2024-09-10T18:54:21.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi131-80444-111", "overlaypool": "test-rados-api-smithi131-80444-146"}]': finished 2024-09-10T18:54:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: osdmap e710: 8 total, 8 up, 8 in 2024-09-10T18:54:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-146", "mode": "writeback"}]: dispatch 2024-09-10T18:54:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: pgmap v1111: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:54:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi131-80444-146", "mode": "writeback"}]': finished 2024-09-10T18:54:21.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:21 smithi175 ceph-mon[33399]: osdmap e711: 8 total, 8 up, 8 in 2024-09-10T18:54:22.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:22 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:54:22.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:22 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:54:22.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:22 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[29369]: osdmap e712: 8 total, 8 up, 8 in 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[29369]: pgmap v1114: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[40885]: osdmap e712: 8 total, 8 up, 8 in 2024-09-10T18:54:23.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:54:23.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-mon[40885]: pgmap v1114: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:54:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:23 smithi175 ceph-mon[33399]: osdmap e712: 8 total, 8 up, 8 in 2024-09-10T18:54:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:23 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:54:23.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:23 smithi175 ceph-mon[33399]: pgmap v1114: 204 pgs: 2 creating+peering, 202 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:24.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:54:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:23] "GET /metrics HTTP/1.1" 200 228532 "" "Prometheus/2.43.0" 2024-09-10T18:54:24.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: osdmap e713: 8 total, 8 up, 8 in 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: osdmap e714: 8 total, 8 up, 8 in 2024-09-10T18:54:24.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:24 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:54:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:54:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: osdmap e713: 8 total, 8 up, 8 in 2024-09-10T18:54:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:54:24.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:54:24.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:54:24.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: osdmap e714: 8 total, 8 up, 8 in 2024-09-10T18:54:24.578 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:54:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:54:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: osdmap e713: 8 total, 8 up, 8 in 2024-09-10T18:54:24.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:54:24.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:54:24.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:54:24.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: osdmap e714: 8 total, 8 up, 8 in 2024-09-10T18:54:24.579 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:24 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T18:54:25.426 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:25 smithi175 ceph-mon[33399]: pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T18:54:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:25 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:54:25.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:25 smithi175 ceph-mon[33399]: osdmap e715: 8 total, 8 up, 8 in 2024-09-10T18:54:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:54:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:25.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[29369]: pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[29369]: osdmap e715: 8 total, 8 up, 8 in 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[40885]: pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T18:54:25.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:25 smithi131 ceph-mon[40885]: osdmap e715: 8 total, 8 up, 8 in 2024-09-10T18:54:26.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:54:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:26 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:54:26.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:26 smithi131 ceph-mon[29369]: osdmap e716: 8 total, 8 up, 8 in 2024-09-10T18:54:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:54:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:26 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:54:26.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:26 smithi131 ceph-mon[40885]: osdmap e716: 8 total, 8 up, 8 in 2024-09-10T18:54:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T18:54:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:26 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T18:54:26.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:26 smithi175 ceph-mon[33399]: osdmap e716: 8 total, 8 up, 8 in 2024-09-10T18:54:27.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:27 smithi131 ceph-mon[29369]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T18:54:27.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:27 smithi131 ceph-mon[40885]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T18:54:27.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:27 smithi175 ceph-mon[33399]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T18:54:28.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:28 smithi131 ceph-mon[29369]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:54:28.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:28 smithi131 ceph-mon[40885]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:54:28.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:28 smithi175 ceph-mon[33399]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T18:54:29.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:29 smithi131 ceph-mon[29369]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:54:29.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:29 smithi131 ceph-mon[40885]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:54:29.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:29 smithi175 ceph-mon[33399]: pgmap v1121: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 0 B/s wr, 3 op/s 2024-09-10T18:54:31.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:31 smithi131 ceph-mon[29369]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:31 smithi131 ceph-mon[40885]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:31.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:31 smithi175 ceph-mon[33399]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 825 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:33.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-mon[29369]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:54:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:33.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-mon[40885]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:54:33.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:33 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:33.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:33 smithi175 ceph-mon[33399]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T18:54:34.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:54:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:33] "GET /metrics HTTP/1.1" 200 228611 "" "Prometheus/2.43.0" 2024-09-10T18:54:34.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:54:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:54:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:54:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:35.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:35.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:35 smithi131 ceph-mon[29369]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:35.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:35 smithi131 ceph-mon[40885]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:35.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:35 smithi175 ceph-mon[33399]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:36.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:36 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:36.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:36 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:36.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:36 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:37.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[29369]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1003 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[29369]: osdmap e717: 8 total, 8 up, 8 in 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[40885]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1003 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[40885]: osdmap e717: 8 total, 8 up, 8 in 2024-09-10T18:54:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:37 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:37.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:37 smithi175 ceph-mon[33399]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1003 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:37 smithi175 ceph-mon[33399]: osdmap e717: 8 total, 8 up, 8 in 2024-09-10T18:54:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:37 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:38.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]': finished 2024-09-10T18:54:38.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[29369]: osdmap e718: 8 total, 8 up, 8 in 2024-09-10T18:54:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:38.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]': finished 2024-09-10T18:54:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[40885]: osdmap e718: 8 total, 8 up, 8 in 2024-09-10T18:54:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:38.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:38 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:38.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]': finished 2024-09-10T18:54:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:38 smithi175 ceph-mon[33399]: osdmap e718: 8 total, 8 up, 8 in 2024-09-10T18:54:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:38.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:38 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-146"}]: dispatch 2024-09-10T18:54:39.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[29369]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[29369]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[29369]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[29369]: osdmap e719: 8 total, 8 up, 8 in 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[40885]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[40885]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[40885]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:54:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:39 smithi131 ceph-mon[40885]: osdmap e719: 8 total, 8 up, 8 in 2024-09-10T18:54:39.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:39 smithi175 ceph-mon[33399]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:54:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:39 smithi175 ceph-mon[33399]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:39 smithi175 ceph-mon[33399]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T18:54:39.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:39 smithi175 ceph-mon[33399]: osdmap e719: 8 total, 8 up, 8 in 2024-09-10T18:54:40.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: osdmap e720: 8 total, 8 up, 8 in 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:40.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: osdmap e720: 8 total, 8 up, 8 in 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:40.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:40.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: osdmap e720: 8 total, 8 up, 8 in 2024-09-10T18:54:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:40.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[29369]: pgmap v1131: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:54:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[29369]: osdmap e721: 8 total, 8 up, 8 in 2024-09-10T18:54:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[40885]: pgmap v1131: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:54:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[40885]: osdmap e721: 8 total, 8 up, 8 in 2024-09-10T18:54:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:41 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:41.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:41 smithi175 ceph-mon[33399]: pgmap v1131: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-09-10T18:54:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:41 smithi175 ceph-mon[33399]: osdmap e721: 8 total, 8 up, 8 in 2024-09-10T18:54:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:41 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:54:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:42 smithi175 ceph-mon[33399]: osdmap e722: 8 total, 8 up, 8 in 2024-09-10T18:54:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:42 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]: dispatch 2024-09-10T18:54:43.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:42 smithi131 ceph-mon[29369]: osdmap e722: 8 total, 8 up, 8 in 2024-09-10T18:54:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:42 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]: dispatch 2024-09-10T18:54:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:54:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:42 smithi131 ceph-mon[40885]: osdmap e722: 8 total, 8 up, 8 in 2024-09-10T18:54:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:42 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]: dispatch 2024-09-10T18:54:44.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:43] "GET /metrics HTTP/1.1" 200 228611 "" "Prometheus/2.43.0" 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[29369]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]': finished 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[29369]: osdmap e723: 8 total, 8 up, 8 in 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]: dispatch 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[40885]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]': finished 2024-09-10T18:54:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[40885]: osdmap e723: 8 total, 8 up, 8 in 2024-09-10T18:54:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:44.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:43 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]: dispatch 2024-09-10T18:54:44.177 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:54:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:54:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:43 smithi175 ceph-mon[33399]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]': finished 2024-09-10T18:54:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:43 smithi175 ceph-mon[33399]: osdmap e723: 8 total, 8 up, 8 in 2024-09-10T18:54:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:43 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-148"}]: dispatch 2024-09-10T18:54:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[29369]: osdmap e724: 8 total, 8 up, 8 in 2024-09-10T18:54:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[29369]: pgmap v1137: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[29369]: osdmap e725: 8 total, 8 up, 8 in 2024-09-10T18:54:45.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:45.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[40885]: osdmap e724: 8 total, 8 up, 8 in 2024-09-10T18:54:45.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:45.080 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[40885]: pgmap v1137: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[40885]: osdmap e725: 8 total, 8 up, 8 in 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:44 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:44 smithi175 ceph-mon[33399]: osdmap e724: 8 total, 8 up, 8 in 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:44 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:44 smithi175 ceph-mon[33399]: pgmap v1137: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:44 smithi175 ceph-mon[33399]: osdmap e725: 8 total, 8 up, 8 in 2024-09-10T18:54:45.081 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:44 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:54:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:46.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:46.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:46 smithi175 ceph-mon[33399]: osdmap e726: 8 total, 8 up, 8 in 2024-09-10T18:54:46.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:46.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-150"}]: dispatch 2024-09-10T18:54:46.677 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:54:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a[67018]: logger=sqlstore.transactions t=2024-09-10T18:54:46.35102007Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:54:47.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[29369]: osdmap e726: 8 total, 8 up, 8 in 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-150"}]: dispatch 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[40885]: osdmap e726: 8 total, 8 up, 8 in 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-150"}]: dispatch 2024-09-10T18:54:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:47 smithi175 ceph-mon[33399]: pgmap v1140: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:47 smithi175 ceph-mon[33399]: osdmap e727: 8 total, 8 up, 8 in 2024-09-10T18:54:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:47 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[29369]: pgmap v1140: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[29369]: osdmap e727: 8 total, 8 up, 8 in 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[40885]: pgmap v1140: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[40885]: osdmap e727: 8 total, 8 up, 8 in 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:54:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:48.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:48 smithi175 ceph-mon[33399]: osdmap e728: 8 total, 8 up, 8 in 2024-09-10T18:54:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:48.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:48 smithi175 ceph-mon[33399]: osdmap e729: 8 total, 8 up, 8 in 2024-09-10T18:54:49.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[29369]: osdmap e728: 8 total, 8 up, 8 in 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[29369]: osdmap e729: 8 total, 8 up, 8 in 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[40885]: osdmap e728: 8 total, 8 up, 8 in 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:49.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:48 smithi131 ceph-mon[40885]: osdmap e729: 8 total, 8 up, 8 in 2024-09-10T18:54:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:49 smithi175 ceph-mon[33399]: pgmap v1143: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:54:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:50.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:49 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-152"}]: dispatch 2024-09-10T18:54:50.178 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:49 smithi175 ceph-mon[33399]: osdmap e730: 8 total, 8 up, 8 in 2024-09-10T18:54:50.326 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[29369]: pgmap v1143: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-152"}]: dispatch 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[29369]: osdmap e730: 8 total, 8 up, 8 in 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[40885]: pgmap v1143: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-152"}]: dispatch 2024-09-10T18:54:50.327 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:49 smithi131 ceph-mon[40885]: osdmap e730: 8 total, 8 up, 8 in 2024-09-10T18:54:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:51 smithi175 ceph-mon[33399]: pgmap v1146: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:54:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:51 smithi175 ceph-mon[33399]: osdmap e731: 8 total, 8 up, 8 in 2024-09-10T18:54:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:51 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:52.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:51 smithi131 ceph-mon[29369]: pgmap v1146: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:54:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:51 smithi131 ceph-mon[29369]: osdmap e731: 8 total, 8 up, 8 in 2024-09-10T18:54:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:51 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:51 smithi131 ceph-mon[40885]: pgmap v1146: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T18:54:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:51 smithi131 ceph-mon[40885]: osdmap e731: 8 total, 8 up, 8 in 2024-09-10T18:54:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:51 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:54:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:52 smithi175 ceph-mon[33399]: osdmap e732: 8 total, 8 up, 8 in 2024-09-10T18:54:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-111","var": "dedup_tier","val": "test-rados-api-smithi131-80444-154"}]: dispatch 2024-09-10T18:54:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:52.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:52 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-154"}]: dispatch 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[29369]: osdmap e732: 8 total, 8 up, 8 in 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-111","var": "dedup_tier","val": "test-rados-api-smithi131-80444-154"}]: dispatch 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-154"}]: dispatch 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi131-80444-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[40885]: osdmap e732: 8 total, 8 up, 8 in 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi131-80444-111","var": "dedup_tier","val": "test-rados-api-smithi131-80444-154"}]: dispatch 2024-09-10T18:54:53.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:53.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:52 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi131-80444-111", "tierpool": "test-rados-api-smithi131-80444-154"}]: dispatch 2024-09-10T18:54:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:53 smithi175 ceph-mon[33399]: pgmap v1149: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:53 smithi175 ceph-mon[33399]: osdmap e733: 8 total, 8 up, 8 in 2024-09-10T18:54:54.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:54:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:53] "GET /metrics HTTP/1.1" 200 226164 "" "Prometheus/2.43.0" 2024-09-10T18:54:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:53 smithi131 ceph-mon[40885]: pgmap v1149: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:53 smithi131 ceph-mon[40885]: osdmap e733: 8 total, 8 up, 8 in 2024-09-10T18:54:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:53 smithi131 ceph-mon[29369]: pgmap v1149: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:54.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:53 smithi131 ceph-mon[29369]: osdmap e733: 8 total, 8 up, 8 in 2024-09-10T18:54:54.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:54:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:54:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:54 smithi175 ceph-mon[33399]: osdmap e734: 8 total, 8 up, 8 in 2024-09-10T18:54:54.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:54 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:55.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:54 smithi131 ceph-mon[29369]: osdmap e734: 8 total, 8 up, 8 in 2024-09-10T18:54:55.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:54 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:54 smithi131 ceph-mon[40885]: osdmap e734: 8 total, 8 up, 8 in 2024-09-10T18:54:55.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:54 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:55.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:54:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:55.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7853 ms) 2024-09-10T18:54:55.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-10T18:54:55.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12321 ms) 2024-09-10T18:54:55.638 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8069 ms) 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13090 ms) 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7530 ms) 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8051 ms) 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: ok, hit_set contains 255:602f83fe:::foo:head 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9040 ms) 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-10T18:54:55.639 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first is 1725994412 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,1725994417,1725994418,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,1725994417,1725994418,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,1725994417,1725994418,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,1725994417,1725994418,1725994420,1725994421,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,1725994417,1725994418,1725994420,1725994421,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994412,1725994414,1725994415,1725994417,1725994418,1725994420,1725994421,0 2024-09-10T18:54:55.640 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: got ls 1725994415,1725994417,1725994418,1725994420,1725994421,1725994423,1725994424,0 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: first now 1725994415, trimmed 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20336 ms) 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: foo0 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14073 ms) 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17518 ms) 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22330 ms) 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5036 ms) 2024-09-10T18:54:55.641 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3011 ms) 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3014 ms) 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3022 ms) 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (229761 ms total) 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (848511 ms total) 2024-09-10T18:54:55.642 INFO:tasks.workunit.client.0.smithi131.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80155 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80155 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80373 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80373 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80924 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80924 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.645 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80649 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80649 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80506 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80506 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80239 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80239 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80562 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80562 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.646 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80258 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80258 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80806 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80806 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80130 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80130 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80193 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80193 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80865 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80865 2024-09-10T18:54:55.647 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80121 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80121 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80141 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80141 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80414 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80414 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80549 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80549 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.648 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80466 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80466 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80892 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80892 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80773 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80773 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80307 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80307 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.649 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80690 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80690 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80114 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80114 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80172 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80172 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80593 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80593 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80213 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80213 2024-09-10T18:54:55.650 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80702 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80702 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=80749 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 80749 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ exit 0 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ cleanup 2024-09-10T18:54:55.651 INFO:tasks.workunit.client.0.smithi131.stderr:+ pkill -P 80109 2024-09-10T18:54:55.655 INFO:tasks.workunit.client.0.smithi131.stderr:+ true 2024-09-10T18:54:55.656 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T18:54:55.656 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T18:54:55.687 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-10T18:54:55.688 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-10T18:54:55.688 DEBUG:teuthology.orchestra.run.smithi131:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-10T18:54:55.749 INFO:tasks.workunit.client.0.smithi131.stderr:++ uuidgen 2024-09-10T18:54:55.754 INFO:tasks.workunit.client.0.smithi131.stderr:+ p=e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:54:55.755 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool create e7116ce6-52bb-4120-b144-3681a246036c 12 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.833+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/3542121480 >> v1:172.21.15.131:6789/0 conn(0x7f8d78101560 legacy=0x7f8d78101930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.833+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/3542121480 shutdown_connections 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.833+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/3542121480 >> 172.21.15.131:0/3542121480 conn(0x7f8d780f6ff0 msgr2=0x7f8d780f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.833+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/3542121480 shutdown_connections 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.833+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/3542121480 wait complete. 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d7d8dd640 1 Processor -- start 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d7d8dd640 1 -- start start 2024-09-10T18:54:55.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d7d8dd640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d7811c030 con 0x7f8d78101560 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d7d8dd640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d7811d1f0 con 0x7f8d78104320 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d7d8dd640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d7811e3f0 con 0x7f8d78107eb0 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d76ffd640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8d78101560 0x7f8d78065c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49202/0 (socket says 172.21.15.131:49202) 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.834+0000 7f8d76ffd640 1 -- 172.21.15.131:0/1750774941 learned_addr learned my addr 172.21.15.131:0/1750774941 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892691561 0 0) 0x7f8d7811c030 con 0x7f8d78101560 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d640034e0 con 0x7f8d78101560 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310373852 0 0) 0x7f8d7811e3f0 con 0x7f8d78107eb0 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d7811c030 con 0x7f8d78107eb0 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706148521 0 0) 0x7f8d7811d1f0 con 0x7f8d78104320 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d7811e3f0 con 0x7f8d78104320 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1672045115 0 0) 0x7f8d7811c030 con 0x7f8d78107eb0 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d7811d1f0 con 0x7f8d78107eb0 2024-09-10T18:54:55.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f8d70001780 con 0x7f8d78107eb0 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2633197476 0 0) 0x7f8d640034e0 con 0x7f8d78101560 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d7811c030 con 0x7f8d78101560 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f8d68002d60 con 0x7f8d78101560 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1863666514 0 0) 0x7f8d7811e3f0 con 0x7f8d78104320 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d640034e0 con 0x7f8d78104320 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f8d600019f0 con 0x7f8d78104320 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3892716289 0 0) 0x7f8d7811d1f0 con 0x7f8d78107eb0 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 >> v1:172.21.15.175:6789/0 conn(0x7f8d78104320 legacy=0x7f8d78066340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 >> v1:172.21.15.131:6789/0 conn(0x7f8d78101560 legacy=0x7f8d78065c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7811f5f0 con 0x7f8d78107eb0 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7811e5e0 con 0x7f8d78107eb0 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7811eab0 con 0x7f8d78107eb0 2024-09-10T18:54:55.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f8d70001e80 con 0x7f8d78107eb0 2024-09-10T18:54:55.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.835+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f8d700030d0 con 0x7f8d78107eb0 2024-09-10T18:54:55.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.836+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d58005010 con 0x7f8d78107eb0 2024-09-10T18:54:55.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.837+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f8d70001a30 con 0x7f8d78107eb0 2024-09-10T18:54:55.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.837+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(736..736 src has 1..736) v4 ==== 7422+0+0 (unknown 1718138966 0 0) 0x7f8d70087a90 con 0x7f8d78107eb0 2024-09-10T18:54:55.843 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:55.842+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f8d70054290 con 0x7f8d78107eb0 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: pgmap v1152: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: osdmap e735: 8 total, 8 up, 8 in 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:55 smithi175 ceph-mon[33399]: osdmap e736: 8 total, 8 up, 8 in 2024-09-10T18:54:56.002 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.001+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12} v 0) v1 -- 0x7f8d58005840 con 0x7f8d78107eb0 2024-09-10T18:54:56.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: pgmap v1152: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: osdmap e735: 8 total, 8 up, 8 in 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[29369]: osdmap e736: 8 total, 8 up, 8 in 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: pgmap v1152: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: osdmap e735: 8 total, 8 up, 8 in 2024-09-10T18:54:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:54:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]: dispatch 2024-09-10T18:54:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3699918358' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi131-80444-111"}]': finished 2024-09-10T18:54:56.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:55 smithi131 ceph-mon[40885]: osdmap e736: 8 total, 8 up, 8 in 2024-09-10T18:54:56.652 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.651+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]=0 pool 'e7116ce6-52bb-4120-b144-3681a246036c' created v737) v1 ==== 176+0+0 (unknown 3881245274 0 0) 0x7f8d70053c30 con 0x7f8d78107eb0 2024-09-10T18:54:56.749 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.747+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12} v 0) v1 -- 0x7f8d580052c0 con 0x7f8d78107eb0 2024-09-10T18:54:56.749 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.748+0000 7f8d6f7fe640 1 -- 172.21.15.131:0/1750774941 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]=0 pool 'e7116ce6-52bb-4120-b144-3681a246036c' already exists v737) v1 ==== 183+0+0 (unknown 3498255436 0 0) 0x7f8d70053e10 con 0x7f8d78107eb0 2024-09-10T18:54:56.750 INFO:tasks.workunit.client.0.smithi131.stderr:pool 'e7116ce6-52bb-4120-b144-3681a246036c' already exists 2024-09-10T18:54:56.752 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.750+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 >> v1:172.21.15.131:6800/336444945 conn(0x7f8d6406ff80 legacy=0x7f8d64072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:56.752 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.750+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 >> v1:172.21.15.131:6790/0 conn(0x7f8d78107eb0 legacy=0x7f8d7811a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:56.752 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.750+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 shutdown_connections 2024-09-10T18:54:56.752 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.750+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 >> 172.21.15.131:0/1750774941 conn(0x7f8d780f6ff0 msgr2=0x7f8d7810b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:56.752 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.750+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 shutdown_connections 2024-09-10T18:54:56.752 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.750+0000 7f8d7d8dd640 1 -- 172.21.15.131:0/1750774941 wait complete. 2024-09-10T18:54:56.760 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_objects 10 2024-09-10T18:54:56.838 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.834+0000 7f970e75d640 1 -- 172.21.15.131:0/1258467668 >> v1:172.21.15.131:6790/0 conn(0x7f97081014c0 legacy=0x7f9708101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.834+0000 7f970e75d640 1 -- 172.21.15.131:0/1258467668 shutdown_connections 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.834+0000 7f970e75d640 1 -- 172.21.15.131:0/1258467668 >> 172.21.15.131:0/1258467668 conn(0x7f97080f6fe0 msgr2=0x7f97080f93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.834+0000 7f970e75d640 1 -- 172.21.15.131:0/1258467668 shutdown_connections 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.834+0000 7f970e75d640 1 -- 172.21.15.131:0/1258467668 wait complete. 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f970e75d640 1 Processor -- start 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f970e75d640 1 -- start start 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f970e75d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f970811e040 con 0x7f9708104230 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f970e75d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f970811f200 con 0x7f9708107d90 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f970e75d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9708120400 con 0x7f97081014c0 2024-09-10T18:54:56.839 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f9707fff640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f97081014c0 0x7f9708106ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:41696/0 (socket says 172.21.15.131:41696) 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f9707fff640 1 -- 172.21.15.131:0/3187630545 learned_addr learned my addr 172.21.15.131:0/3187630545 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f97077fe640 1 --1- 172.21.15.131:0/3187630545 >> v1:172.21.15.131:6789/0 conn(0x7f9708104230 0x7f9708107600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:49220/0 (socket says 172.21.15.131:49220) 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415260776 0 0) 0x7f970811e040 con 0x7f9708104230 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f4003120 con 0x7f9708104230 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.835+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2799397760 0 0) 0x7f9708120400 con 0x7f97081014c0 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f970811e040 con 0x7f97081014c0 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549407648 0 0) 0x7f970811f200 con 0x7f9708107d90 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9708120400 con 0x7f9708107d90 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3631780550 0 0) 0x7f96f4003120 con 0x7f9708104230 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f970811f200 con 0x7f9708104230 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2097514696 0 0) 0x7f970811e040 con 0x7f97081014c0 2024-09-10T18:54:56.840 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f96f4003120 con 0x7f97081014c0 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f96f00018c0 con 0x7f9708104230 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 615588678 0 0) 0x7f9708120400 con 0x7f9708107d90 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f970811e040 con 0x7f9708107d90 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f96f8002b20 con 0x7f97081014c0 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f97000019a0 con 0x7f9708107d90 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3445590771 0 0) 0x7f970811f200 con 0x7f9708104230 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 >> v1:172.21.15.131:6790/0 conn(0x7f97081014c0 legacy=0x7f9708106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 >> v1:172.21.15.175:6789/0 conn(0x7f9708107d90 legacy=0x7f970811c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9708121600 con 0x7f9708104230 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97081205f0 con 0x7f9708104230 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9708120ac0 con 0x7f9708104230 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f96f0002c80 con 0x7f9708104230 2024-09-10T18:54:56.841 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.836+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f96f0005a10 con 0x7f9708104230 2024-09-10T18:54:56.842 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.837+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e8005010 con 0x7f9708104230 2024-09-10T18:54:56.842 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.838+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f96f0001de0 con 0x7f9708104230 2024-09-10T18:54:56.842 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.838+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(737..737 src has 1..737) v4 ==== 7797+0+0 (unknown 540333448 0 0) 0x7f96f008a440 con 0x7f9708104230 2024-09-10T18:54:56.843 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.842+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f96f0056b30 con 0x7f9708104230 2024-09-10T18:54:56.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:56 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1750774941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:56.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:56 smithi175 ceph-mon[33399]: from='client.57158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.000 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:56.998+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f96e80052c0 con 0x7f9708104230 2024-09-10T18:54:57.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:56 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1750774941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:56 smithi131 ceph-mon[29369]: from='client.57158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:56 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1750774941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:56 smithi131 ceph-mon[40885]: from='client.57158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:57.653+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool e7116ce6-52bb-4120-b144-3681a246036c v738) v1 ==== 223+0+0 (unknown 1268303695 0 0) 0x7f96f00564d0 con 0x7f9708104230 2024-09-10T18:54:57.755 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:57.753+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f96e8001090 con 0x7f9708104230 2024-09-10T18:54:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:57 smithi175 ceph-mon[33399]: pgmap v1155: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:57 smithi175 ceph-mon[33399]: from='client.57158 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]': finished 2024-09-10T18:54:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:57 smithi175 ceph-mon[33399]: osdmap e737: 8 total, 8 up, 8 in 2024-09-10T18:54:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1750774941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:57 smithi175 ceph-mon[33399]: from='client.57158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:57.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:57 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:58.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[29369]: pgmap v1155: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:58.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[29369]: from='client.57158 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]': finished 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[29369]: osdmap e737: 8 total, 8 up, 8 in 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1750774941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[29369]: from='client.57158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[40885]: pgmap v1155: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[40885]: from='client.57158 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]': finished 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[40885]: osdmap e737: 8 total, 8 up, 8 in 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1750774941' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[40885]: from='client.57158 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pg_num": 12}]: dispatch 2024-09-10T18:54:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:57 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:58.667 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.666+0000 7f9704ff9640 1 -- 172.21.15.131:0/3187630545 <== mon.0 v1:172.21.15.131:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool e7116ce6-52bb-4120-b144-3681a246036c v739) v1 ==== 223+0+0 (unknown 48063039 0 0) 0x7f96f00566b0 con 0x7f9708104230 2024-09-10T18:54:58.667 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 10 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:54:58.670 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.669+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 >> v1:172.21.15.131:6800/336444945 conn(0x7f96f406fd50 legacy=0x7f96f4072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:58.670 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.669+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 >> v1:172.21.15.131:6789/0 conn(0x7f9708104230 legacy=0x7f9708107600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:58.670 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.669+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 shutdown_connections 2024-09-10T18:54:58.670 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.669+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 >> 172.21.15.131:0/3187630545 conn(0x7f97080f6fe0 msgr2=0x7f970810ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:58.671 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.669+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 shutdown_connections 2024-09-10T18:54:58.671 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.669+0000 7f970e75d640 1 -- 172.21.15.131:0/3187630545 wait complete. 2024-09-10T18:54:58.681 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool application enable e7116ce6-52bb-4120-b144-3681a246036c rados 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.761+0000 7fd249700640 1 -- 172.21.15.131:0/3454413031 >> v1:172.21.15.131:6790/0 conn(0x7fd24410b120 legacy=0x7fd24410d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.761+0000 7fd249700640 1 -- 172.21.15.131:0/3454413031 shutdown_connections 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.761+0000 7fd249700640 1 -- 172.21.15.131:0/3454413031 >> 172.21.15.131:0/3454413031 conn(0x7fd2440f7020 msgr2=0x7fd2440f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.761+0000 7fd249700640 1 -- 172.21.15.131:0/3454413031 shutdown_connections 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.761+0000 7fd249700640 1 -- 172.21.15.131:0/3454413031 wait complete. 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd249700640 1 Processor -- start 2024-09-10T18:54:58.763 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd249700640 1 -- start start 2024-09-10T18:54:58.766 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd249700640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd24411e0c0 con 0x7fd24410b120 2024-09-10T18:54:58.766 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd249700640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd24411f2c0 con 0x7fd2440fbdd0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd249700640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd2441204c0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd242ffd640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7fd2440fb2f0 0x7fd244107280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:41706/0 (socket says 172.21.15.131:41706) 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd242ffd640 1 -- 172.21.15.131:0/2135112112 learned_addr learned my addr 172.21.15.131:0/2135112112 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105412369 0 0) 0x7fd2441204c0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003120 con 0x7fd2440fb2f0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658323162 0 0) 0x7fd24411e0c0 con 0x7fd24410b120 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2441204c0 con 0x7fd24410b120 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3598099592 0 0) 0x7fd24411f2c0 con 0x7fd2440fbdd0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.762+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd24411e0c0 con 0x7fd2440fbdd0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2331451328 0 0) 0x7fd22c003120 con 0x7fd2440fb2f0 2024-09-10T18:54:58.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd24411f2c0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fd234001d70 con 0x7fd2440fb2f0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1330259059 0 0) 0x7fd24411e0c0 con 0x7fd2440fbdd0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd22c003120 con 0x7fd2440fbdd0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3210656051 0 0) 0x7fd2441204c0 con 0x7fd24410b120 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd24411e0c0 con 0x7fd24410b120 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fd238001900 con 0x7fd2440fbdd0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fd23c001350 con 0x7fd24410b120 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1019345382 0 0) 0x7fd24411f2c0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 >> v1:172.21.15.175:6789/0 conn(0x7fd2440fbdd0 legacy=0x7fd2441079b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 >> v1:172.21.15.131:6789/0 conn(0x7fd24410b120 legacy=0x7fd24411c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2441216c0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2441206b0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244120b00 con 0x7fd2440fb2f0 2024-09-10T18:54:58.769 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7fd234002760 con 0x7fd2440fb2f0 2024-09-10T18:54:58.769 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.763+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7fd234005cd0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.769 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.764+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd230005010 con 0x7fd2440fb2f0 2024-09-10T18:54:58.769 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.764+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7fd234005430 con 0x7fd2440fb2f0 2024-09-10T18:54:58.769 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.765+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(739..739 src has 1..739) v4 ==== 7797+0+0 (unknown 861320463 0 0) 0x7fd23408a810 con 0x7fd2440fb2f0 2024-09-10T18:54:58.770 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.769+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7fd234056f00 con 0x7fd2440fb2f0 2024-09-10T18:54:58.925 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:58.924+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"} v 0) v1 -- 0x7fd2300052c0 con 0x7fd2440fb2f0 2024-09-10T18:54:58.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:58.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:58 smithi175 ceph-mon[33399]: osdmap e738: 8 total, 8 up, 8 in 2024-09-10T18:54:58.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:58 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:59.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:59.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:58 smithi131 ceph-mon[29369]: osdmap e738: 8 total, 8 up, 8 in 2024-09-10T18:54:59.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:58 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:54:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:58 smithi131 ceph-mon[40885]: osdmap e738: 8 total, 8 up, 8 in 2024-09-10T18:54:59.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:58 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:54:59.673 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:59.672+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]=0 enabled application 'rados' on pool 'e7116ce6-52bb-4120-b144-3681a246036c' v740) v1 ==== 213+0+0 (unknown 4064113846 0 0) 0x7fd2340568a0 con 0x7fd2440fb2f0 2024-09-10T18:54:59.772 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:54:59.771+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"} v 0) v1 -- 0x7fd230001090 con 0x7fd2440fb2f0 2024-09-10T18:55:00.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[29369]: pgmap v1158: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[29369]: osdmap e739: 8 total, 8 up, 8 in 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2135112112' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[29369]: from='client.57170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[40885]: pgmap v1158: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[40885]: osdmap e739: 8 total, 8 up, 8 in 2024-09-10T18:55:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2135112112' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:00.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:54:59 smithi131 ceph-mon[40885]: from='client.57170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:59 smithi175 ceph-mon[33399]: pgmap v1158: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3187630545' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:55:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:59 smithi175 ceph-mon[33399]: osdmap e739: 8 total, 8 up, 8 in 2024-09-10T18:55:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:59 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2135112112' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:00.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:54:59 smithi175 ceph-mon[33399]: from='client.57170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:00.687 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.686+0000 7fd22b7fe640 1 -- 172.21.15.131:0/2135112112 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]=0 enabled application 'rados' on pool 'e7116ce6-52bb-4120-b144-3681a246036c' v741) v1 ==== 213+0+0 (unknown 3357311568 0 0) 0x7fd234056a80 con 0x7fd2440fb2f0 2024-09-10T18:55:00.687 INFO:tasks.workunit.client.0.smithi131.stderr:enabled application 'rados' on pool 'e7116ce6-52bb-4120-b144-3681a246036c' 2024-09-10T18:55:00.690 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.689+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 >> v1:172.21.15.131:6800/336444945 conn(0x7fd22c06fe40 legacy=0x7fd22c0722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:00.690 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.689+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 >> v1:172.21.15.131:6790/0 conn(0x7fd2440fb2f0 legacy=0x7fd244107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:00.691 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.689+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 shutdown_connections 2024-09-10T18:55:00.691 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.689+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 >> 172.21.15.131:0/2135112112 conn(0x7fd2440f7020 msgr2=0x7fd24410ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:00.691 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.689+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 shutdown_connections 2024-09-10T18:55:00.691 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:00.689+0000 7fd249700640 1 -- 172.21.15.131:0/2135112112 wait complete. 2024-09-10T18:55:00.701 INFO:tasks.workunit.client.0.smithi131.stderr:++ seq 1 10 2024-09-10T18:55:00.712 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.712 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj1 /etc/passwd 2024-09-10T18:55:00.744 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.744 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj2 /etc/passwd 2024-09-10T18:55:00.778 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.778 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj3 /etc/passwd 2024-09-10T18:55:00.809 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.809 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj4 /etc/passwd 2024-09-10T18:55:00.837 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.838 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj5 /etc/passwd 2024-09-10T18:55:00.866 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.866 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj6 /etc/passwd 2024-09-10T18:55:00.894 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.894 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj7 /etc/passwd 2024-09-10T18:55:00.923 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.923 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj8 /etc/passwd 2024-09-10T18:55:00.951 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.951 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj9 /etc/passwd 2024-09-10T18:55:00.980 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:55:00.980 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put obj10 /etc/passwd 2024-09-10T18:55:01.008 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-09-10T18:55:01.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[29369]: from='client.57170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]': finished 2024-09-10T18:55:01.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[29369]: osdmap e740: 8 total, 8 up, 8 in 2024-09-10T18:55:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2135112112' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:01.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[29369]: from='client.57170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[40885]: from='client.57170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]': finished 2024-09-10T18:55:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[40885]: osdmap e740: 8 total, 8 up, 8 in 2024-09-10T18:55:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2135112112' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:01.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:00 smithi131 ceph-mon[40885]: from='client.57170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:01.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:00 smithi175 ceph-mon[33399]: from='client.57170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]': finished 2024-09-10T18:55:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:00 smithi175 ceph-mon[33399]: osdmap e740: 8 total, 8 up, 8 in 2024-09-10T18:55:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:00 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2135112112' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:01.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:00 smithi175 ceph-mon[33399]: from='client.57170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]: dispatch 2024-09-10T18:55:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[29369]: pgmap v1161: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[29369]: from='client.57170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]': finished 2024-09-10T18:55:01.978 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[29369]: osdmap e741: 8 total, 8 up, 8 in 2024-09-10T18:55:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[40885]: pgmap v1161: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[40885]: from='client.57170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]': finished 2024-09-10T18:55:01.979 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:01 smithi131 ceph-mon[40885]: osdmap e741: 8 total, 8 up, 8 in 2024-09-10T18:55:02.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:01 smithi175 ceph-mon[33399]: pgmap v1161: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:01 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:01 smithi175 ceph-mon[33399]: from='client.57170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "app": "rados"}]': finished 2024-09-10T18:55:02.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:01 smithi175 ceph-mon[33399]: osdmap e741: 8 total, 8 up, 8 in 2024-09-10T18:55:02.995 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:02.995 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:03.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:04.064 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:03 smithi175 ceph-mon[33399]: pgmap v1163: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:55:04.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:03 smithi131 ceph-mon[29369]: pgmap v1163: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:55:04.076 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:03 smithi131 ceph-mon[40885]: pgmap v1163: 176 pgs: 176 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T18:55:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:55:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:03] "GET /metrics HTTP/1.1" 200 226113 "" "Prometheus/2.43.0" 2024-09-10T18:55:04.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:55:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:55:05.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:55:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:05 smithi131 ceph-mon[29369]: pgmap v1164: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T18:55:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:05 smithi131 ceph-mon[40885]: pgmap v1164: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T18:55:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:06.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:05 smithi175 ceph-mon[33399]: pgmap v1164: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T18:55:08.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[29369]: pgmap v1165: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 655 B/s rd, 17 KiB/s wr, 1 op/s 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[29369]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' is full (reached quota's max_objects: 10) 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[29369]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[29369]: osdmap e742: 8 total, 8 up, 8 in 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[40885]: pgmap v1165: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 655 B/s rd, 17 KiB/s wr, 1 op/s 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[40885]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' is full (reached quota's max_objects: 10) 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[40885]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:07 smithi131 ceph-mon[40885]: osdmap e742: 8 total, 8 up, 8 in 2024-09-10T18:55:08.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:07 smithi175 ceph-mon[33399]: pgmap v1165: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 655 B/s rd, 17 KiB/s wr, 1 op/s 2024-09-10T18:55:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:07 smithi175 ceph-mon[33399]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' is full (reached quota's max_objects: 10) 2024-09-10T18:55:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:07 smithi175 ceph-mon[33399]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:07 smithi175 ceph-mon[33399]: osdmap e742: 8 total, 8 up, 8 in 2024-09-10T18:55:10.076 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:09 smithi131 ceph-mon[40885]: pgmap v1167: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-10T18:55:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:09 smithi131 ceph-mon[29369]: pgmap v1167: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-10T18:55:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:09 smithi175 ceph-mon[33399]: pgmap v1167: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 16 KiB/s wr, 2 op/s 2024-09-10T18:55:12.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:11 smithi131 ceph-mon[29369]: pgmap v1168: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-09-10T18:55:12.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:11 smithi131 ceph-mon[40885]: pgmap v1168: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-09-10T18:55:12.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:11 smithi175 ceph-mon[33399]: pgmap v1168: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 13 KiB/s wr, 2 op/s 2024-09-10T18:55:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:13 smithi131 ceph-mon[29369]: pgmap v1169: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:55:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:13 smithi131 ceph-mon[40885]: pgmap v1169: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:55:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:13 smithi175 ceph-mon[33399]: pgmap v1169: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:55:14.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:55:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:13] "GET /metrics HTTP/1.1" 200 226113 "" "Prometheus/2.43.0" 2024-09-10T18:55:14.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:55:14 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:55:15.328 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:55:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:15 smithi175 ceph-mon[33399]: pgmap v1170: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:15.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:15 smithi131 ceph-mon[29369]: pgmap v1170: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:15.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:15.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:15 smithi131 ceph-mon[40885]: pgmap v1170: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:17 smithi131 ceph-mon[29369]: pgmap v1171: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:17 smithi131 ceph-mon[40885]: pgmap v1171: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:17 smithi175 ceph-mon[33399]: pgmap v1171: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:19.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:19 smithi131 ceph-mon[29369]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:55:19.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:19 smithi131 ceph-mon[40885]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:55:19.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:19 smithi175 ceph-mon[33399]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:55:21.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:21 smithi131 ceph-mon[29369]: pgmap v1173: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:21 smithi131 ceph-mon[40885]: pgmap v1173: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:21.929 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:21 smithi175 ceph-mon[33399]: pgmap v1173: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:23.805 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:23 smithi131 ceph-mon[29369]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:23.805 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:23 smithi131 ceph-mon[40885]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:23.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:23 smithi175 ceph-mon[33399]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:24.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:55:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:23] "GET /metrics HTTP/1.1" 200 226191 "" "Prometheus/2.43.0" 2024-09-10T18:55:24.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:55:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:55:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:55:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:25.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:25 smithi131 ceph-mon[29369]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:25 smithi131 ceph-mon[40885]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:25 smithi175 ceph-mon[33399]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:27.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:27 smithi131 ceph-mon[29369]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:27 smithi131 ceph-mon[40885]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:27.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:27 smithi175 ceph-mon[33399]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:29.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:29 smithi131 ceph-mon[29369]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:29 smithi131 ceph-mon[40885]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:29.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:29 smithi175 ceph-mon[33399]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:31.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=149201 2024-09-10T18:55:31.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_objects 100 2024-09-10T18:55:31.010 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put onemore /etc/passwd 2024-09-10T18:55:31.101 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 -- 172.21.15.131:0/2515086467 >> v1:172.21.15.175:6789/0 conn(0x7f6a00104220 legacy=0x7f6a00106600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:31.101 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 -- 172.21.15.131:0/2515086467 shutdown_connections 2024-09-10T18:55:31.101 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 -- 172.21.15.131:0/2515086467 >> 172.21.15.131:0/2515086467 conn(0x7f6a000f6fd0 msgr2=0x7f6a000f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 -- 172.21.15.131:0/2515086467 shutdown_connections 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 -- 172.21.15.131:0/2515086467 wait complete. 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 Processor -- start 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.096+0000 7f6a0744f640 1 -- start start 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f6a0744f640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6a0011bdf0 con 0x7f6a00107d80 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f6a0744f640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6a0011cff0 con 0x7f6a001014b0 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f6a0744f640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6a0011e1f0 con 0x7f6a00104220 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f6a049ea640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f6a00104220 0x7f6a00066320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:38006/0 (socket says 172.21.15.131:38006) 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f6a049ea640 1 -- 172.21.15.131:0/2006274274 learned_addr learned my addr 172.21.15.131:0/2006274274 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f6a059ec640 1 --1- 172.21.15.131:0/2006274274 >> v1:172.21.15.131:6789/0 conn(0x7f6a00107d80 0x7f6a0011a4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51644/0 (socket says 172.21.15.131:51644) 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1999683188 0 0) 0x7f6a0011e1f0 con 0x7f6a00104220 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ec0034e0 con 0x7f6a00104220 2024-09-10T18:55:31.102 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1343628533 0 0) 0x7f6a0011bdf0 con 0x7f6a00107d80 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a0011e1f0 con 0x7f6a00107d80 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477871334 0 0) 0x7f6a0011cff0 con 0x7f6a001014b0 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a0011bdf0 con 0x7f6a001014b0 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1176111635 0 0) 0x7f69ec0034e0 con 0x7f6a00104220 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a0011cff0 con 0x7f6a00104220 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f69f00029e0 con 0x7f6a00104220 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1625664627 0 0) 0x7f6a0011e1f0 con 0x7f6a00107d80 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f69ec0034e0 con 0x7f6a00107d80 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f69f8001d10 con 0x7f6a00107d80 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 117792042 0 0) 0x7f6a0011bdf0 con 0x7f6a001014b0 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6a0011e1f0 con 0x7f6a001014b0 2024-09-10T18:55:31.103 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f69fc0019d0 con 0x7f6a001014b0 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4029690393 0 0) 0x7f6a0011cff0 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.097+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 >> v1:172.21.15.175:6789/0 conn(0x7f6a001014b0 legacy=0x7f6a00065c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.098+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 >> v1:172.21.15.131:6789/0 conn(0x7f6a00107d80 legacy=0x7f6a0011a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.098+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a0011f3f0 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.098+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a0011bfe0 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.098+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f69f0002c80 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.098+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f69f0003610 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.098+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0011c430 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.099+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a000fcbb0 con 0x7f6a00104220 2024-09-10T18:55:31.104 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.099+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f69f00038c0 con 0x7f6a00104220 2024-09-10T18:55:31.105 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.100+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(742..742 src has 1..742) v4 ==== 7810+0+0 (unknown 492210812 0 0) 0x7f69f008a790 con 0x7f6a00104220 2024-09-10T18:55:31.105 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.100+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=743}) v3 -- 0x7f6a0011cff0 con 0x7f6a00104220 2024-09-10T18:55:31.106 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.104+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f69f0056e70 con 0x7f6a00104220 2024-09-10T18:55:31.263 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.260+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f6a000ffbe0 con 0x7f6a00104220 2024-09-10T18:55:31.556 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.554+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 10 ==== osd_map(743..743 src has 1..743) v4 ==== 620+0+0 (unknown 2605396153 0 0) 0x7f69f0056bb0 con 0x7f6a00104220 2024-09-10T18:55:31.556 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.554+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=744}) v3 -- 0x7f69ec0034e0 con 0x7f6a00104220 2024-09-10T18:55:31.561 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.558+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool e7116ce6-52bb-4120-b144-3681a246036c v743) v1 ==== 225+0+0 (unknown 1010501578 0 0) 0x7f69f0056810 con 0x7f6a00104220 2024-09-10T18:55:31.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:31.653+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f6a0010ea20 con 0x7f6a00104220 2024-09-10T18:55:31.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:31 smithi131 ceph-mon[29369]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:55:31.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:31 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2006274274' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:31.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:31 smithi131 ceph-mon[29369]: from='client.57275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:31 smithi131 ceph-mon[40885]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:55:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:31 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2006274274' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:31.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:31 smithi131 ceph-mon[40885]: from='client.57275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:31 smithi175 ceph-mon[33399]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:55:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:31 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2006274274' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:31 smithi175 ceph-mon[33399]: from='client.57275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.199 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.198+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 12 ==== osd_map(744..744 src has 1..744) v4 ==== 620+0+0 (unknown 1326881787 0 0) 0x7f69f00577a0 con 0x7f6a00104220 2024-09-10T18:55:32.200 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.198+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=745}) v3 -- 0x7f6a0011bdf0 con 0x7f6a00104220 2024-09-10T18:55:32.203 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.201+0000 7f69f5ffb640 1 -- 172.21.15.131:0/2006274274 <== mon.2 v1:172.21.15.131:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool e7116ce6-52bb-4120-b144-3681a246036c v744) v1 ==== 225+0+0 (unknown 2758223763 0 0) 0x7f69f00569f0 con 0x7f6a00104220 2024-09-10T18:55:32.203 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 100 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:55:32.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.205+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 >> v1:172.21.15.131:6800/336444945 conn(0x7f69ec06fdd0 legacy=0x7f69ec072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:32.206 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.205+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 >> v1:172.21.15.131:6790/0 conn(0x7f6a00104220 legacy=0x7f6a00066320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:32.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.205+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 shutdown_connections 2024-09-10T18:55:32.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.205+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 >> 172.21.15.131:0/2006274274 conn(0x7f6a000f6fd0 msgr2=0x7f6a000fbd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:32.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.205+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 shutdown_connections 2024-09-10T18:55:32.207 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.205+0000 7f6a0744f640 1 -- 172.21.15.131:0/2006274274 wait complete. 2024-09-10T18:55:32.216 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 149201 2024-09-10T18:55:32.216 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 0 -ne 0 ']' 2024-09-10T18:55:32.217 INFO:tasks.workunit.client.0.smithi131.stderr:+ true 2024-09-10T18:55:32.217 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put twomore /etc/passwd 2024-09-10T18:55:32.247 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_bytes 100 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.325+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1606267211 >> v1:172.21.15.131:6790/0 conn(0x7f3b14108e90 legacy=0x7f3b1410b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.325+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1606267211 shutdown_connections 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.325+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1606267211 >> 172.21.15.131:0/1606267211 conn(0x7f3b140f6fd0 msgr2=0x7f3b140f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.325+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1606267211 shutdown_connections 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.325+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1606267211 wait complete. 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b18b9d640 1 Processor -- start 2024-09-10T18:55:32.330 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b18b9d640 1 -- start start 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b18b9d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3b1411c390 con 0x7f3b140fd560 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b18b9d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3b1411d550 con 0x7f3b14108e90 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b18b9d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3b1411e750 con 0x7f3b141185e0 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b1259c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f3b140fd560 0x7f3b14104060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:51678/0 (socket says 172.21.15.131:51678) 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b1259c640 1 -- 172.21.15.131:0/1903494555 learned_addr learned my addr 172.21.15.131:0/1903494555 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b12d9d640 1 --1- 172.21.15.131:0/1903494555 >> v1:172.21.15.131:6790/0 conn(0x7f3b141185e0 0x7f3b1411aa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:38040/0 (socket says 172.21.15.131:38040) 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679484903 0 0) 0x7f3b1411c390 con 0x7f3b140fd560 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3afc0034e0 con 0x7f3b140fd560 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730660322 0 0) 0x7f3b1411e750 con 0x7f3b141185e0 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.326+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b1411c390 con 0x7f3b141185e0 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2637634048 0 0) 0x7f3b1411d550 con 0x7f3b14108e90 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b1411e750 con 0x7f3b14108e90 2024-09-10T18:55:32.331 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2608325723 0 0) 0x7f3b1411c390 con 0x7f3b141185e0 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b1411d550 con 0x7f3b141185e0 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2099114518 0 0) 0x7f3afc0034e0 con 0x7f3b140fd560 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3b1411c390 con 0x7f3b140fd560 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3b0c0017b0 con 0x7f3b141185e0 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3b040018c0 con 0x7f3b140fd560 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3578426364 0 0) 0x7f3b1411e750 con 0x7f3b14108e90 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3afc0034e0 con 0x7f3b14108e90 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3b08001620 con 0x7f3b14108e90 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 334136009 0 0) 0x7f3b1411d550 con 0x7f3b141185e0 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 >> v1:172.21.15.175:6789/0 conn(0x7f3b14108e90 legacy=0x7f3b14104770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 >> v1:172.21.15.131:6789/0 conn(0x7f3b140fd560 legacy=0x7f3b14104060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:32.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1411f950 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1411e940 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1411ed90 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f3b0c002920 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.327+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f3b0c0038d0 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.328+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af4005010 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.328+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f3b0c0023a0 con 0x7f3b141185e0 2024-09-10T18:55:32.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.329+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(744..744 src has 1..744) v4 ==== 7810+0+0 (unknown 906749532 0 0) 0x7f3b0c08aa70 con 0x7f3b141185e0 2024-09-10T18:55:32.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.333+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f3b0c057150 con 0x7f3b141185e0 2024-09-10T18:55:32.490 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:32.488+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f3af40052c0 con 0x7f3b141185e0 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: osdmap e743: 8 total, 8 up, 8 in 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2006274274' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='client.57275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: osdmap e744: 8 total, 8 up, 8 in 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:32.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1903494555' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[29369]: from='client.57290 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: osdmap e743: 8 total, 8 up, 8 in 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2006274274' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='client.57275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: osdmap e744: 8 total, 8 up, 8 in 2024-09-10T18:55:32.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:32.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:32.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1903494555' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:32.829 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:32 smithi131 ceph-mon[40885]: from='client.57290 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: osdmap e743: 8 total, 8 up, 8 in 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2006274274' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='client.57275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='client.57275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "100"}]': finished 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: osdmap e744: 8 total, 8 up, 8 in 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:32.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:32.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1903494555' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:32.928 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:32 smithi175 ceph-mon[33399]: from='client.57290 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:33.278 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:33.277+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool e7116ce6-52bb-4120-b144-3681a246036c v745) v1 ==== 221+0+0 (unknown 3655927378 0 0) 0x7f3b0c056af0 con 0x7f3b141185e0 2024-09-10T18:55:33.375 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:33.374+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f3af4000be0 con 0x7f3b141185e0 2024-09-10T18:55:33.826 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:33] "GET /metrics HTTP/1.1" 200 226152 "" "Prometheus/2.43.0" 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[29369]: pgmap v1181: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[29369]: from='client.57290 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[29369]: osdmap e745: 8 total, 8 up, 8 in 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1903494555' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[29369]: from='client.57290 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[40885]: pgmap v1181: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[40885]: from='client.57290 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:55:33.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[40885]: osdmap e745: 8 total, 8 up, 8 in 2024-09-10T18:55:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1903494555' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:33.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:33 smithi131 ceph-mon[40885]: from='client.57290 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:33 smithi175 ceph-mon[33399]: pgmap v1181: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:55:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:33 smithi175 ceph-mon[33399]: from='client.57290 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:55:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:33 smithi175 ceph-mon[33399]: osdmap e745: 8 total, 8 up, 8 in 2024-09-10T18:55:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:33 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1903494555' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:33.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:33 smithi175 ceph-mon[33399]: from='client.57290 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T18:55:34.289 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.288+0000 7f3b02ffd640 1 -- 172.21.15.131:0/1903494555 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool e7116ce6-52bb-4120-b144-3681a246036c v746) v1 ==== 221+0+0 (unknown 1559495296 0 0) 0x7f3b0c056cd0 con 0x7f3b141185e0 2024-09-10T18:55:34.289 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_bytes = 100 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:55:34.291 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.290+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 >> v1:172.21.15.131:6800/336444945 conn(0x7f3afc06fe30 legacy=0x7f3afc0722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:34.291 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.290+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 >> v1:172.21.15.131:6790/0 conn(0x7f3b141185e0 legacy=0x7f3b1411aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:55:34.292 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.290+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 shutdown_connections 2024-09-10T18:55:34.292 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.290+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 >> 172.21.15.131:0/1903494555 conn(0x7f3b140f6fd0 msgr2=0x7f3b1410c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:55:34.292 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.290+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 shutdown_connections 2024-09-10T18:55:34.292 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:55:34.290+0000 7f3b18b9d640 1 -- 172.21.15.131:0/1903494555 wait complete. 2024-09-10T18:55:34.300 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-09-10T18:55:34.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:55:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:55:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:55:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:35 smithi175 ceph-mon[33399]: from='client.57290 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:55:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:35 smithi175 ceph-mon[33399]: osdmap e746: 8 total, 8 up, 8 in 2024-09-10T18:55:35.428 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:35 smithi175 ceph-mon[33399]: pgmap v1184: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-10T18:55:35.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[29369]: from='client.57290 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[29369]: osdmap e746: 8 total, 8 up, 8 in 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[29369]: pgmap v1184: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[40885]: from='client.57290 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[40885]: osdmap e746: 8 total, 8 up, 8 in 2024-09-10T18:55:35.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:35 smithi131 ceph-mon[40885]: pgmap v1184: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[29369]: pgmap v1185: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[29369]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' is full (reached quota's max_bytes: 100 B) 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[29369]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[29369]: osdmap e747: 8 total, 8 up, 8 in 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[40885]: pgmap v1185: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[40885]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' is full (reached quota's max_bytes: 100 B) 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[40885]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:37.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:37 smithi131 ceph-mon[40885]: osdmap e747: 8 total, 8 up, 8 in 2024-09-10T18:55:37.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:37 smithi175 ceph-mon[33399]: pgmap v1185: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-10T18:55:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:37 smithi175 ceph-mon[33399]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' is full (reached quota's max_bytes: 100 B) 2024-09-10T18:55:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:37 smithi175 ceph-mon[33399]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:37.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:37 smithi175 ceph-mon[33399]: osdmap e747: 8 total, 8 up, 8 in 2024-09-10T18:55:39.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:39 smithi131 ceph-mon[29369]: pgmap v1187: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:55:39.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:39 smithi131 ceph-mon[40885]: pgmap v1187: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:55:39.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:39 smithi175 ceph-mon[33399]: pgmap v1187: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T18:55:40.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:40 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:40.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:40 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:40.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:40 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[29369]: pgmap v1188: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-09-10T18:55:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:41.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[40885]: pgmap v1188: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-09-10T18:55:41.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:41.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:41 smithi175 ceph-mon[33399]: pgmap v1188: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 3.6 KiB/s wr, 1 op/s 2024-09-10T18:55:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:43.818 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:43 smithi131 ceph-mon[29369]: pgmap v1189: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 624 B/s rd, 0 op/s 2024-09-10T18:55:43.818 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:43 smithi131 ceph-mon[40885]: pgmap v1189: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 624 B/s rd, 0 op/s 2024-09-10T18:55:43.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:43 smithi175 ceph-mon[33399]: pgmap v1189: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 624 B/s rd, 0 op/s 2024-09-10T18:55:44.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:55:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:43] "GET /metrics HTTP/1.1" 200 226152 "" "Prometheus/2.43.0" 2024-09-10T18:55:44.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:55:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:55:45.327 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:55:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:45.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:45.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:45 smithi175 ceph-mon[33399]: pgmap v1190: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:45.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:45 smithi131 ceph-mon[29369]: pgmap v1190: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:45.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:45.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:45 smithi131 ceph-mon[40885]: pgmap v1190: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:47.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:47 smithi131 ceph-mon[29369]: pgmap v1191: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:47.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:47 smithi131 ceph-mon[40885]: pgmap v1191: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:47.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:47.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:47 smithi175 ceph-mon[33399]: pgmap v1191: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:55:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:55:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:49.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:49 smithi131 ceph-mon[29369]: pgmap v1192: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:55:49.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:49 smithi131 ceph-mon[40885]: pgmap v1192: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:55:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:49 smithi175 ceph-mon[33399]: pgmap v1192: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:55:51.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:51 smithi131 ceph-mon[29369]: pgmap v1193: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:51.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:51 smithi131 ceph-mon[40885]: pgmap v1193: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:51 smithi175 ceph-mon[33399]: pgmap v1193: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:53.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:53 smithi131 ceph-mon[29369]: pgmap v1194: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:53.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:55:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:53] "GET /metrics HTTP/1.1" 200 226182 "" "Prometheus/2.43.0" 2024-09-10T18:55:53.828 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:53 smithi131 ceph-mon[40885]: pgmap v1194: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:53 smithi175 ceph-mon[33399]: pgmap v1194: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:54.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:55:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:55:55.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:55:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:55:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:55.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:55.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:55 smithi131 ceph-mon[29369]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:55.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:55.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:55 smithi131 ceph-mon[40885]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:55.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:55.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:55 smithi175 ceph-mon[33399]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:57.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:57 smithi131 ceph-mon[29369]: pgmap v1196: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:57.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:57 smithi131 ceph-mon[40885]: pgmap v1196: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:57.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:57 smithi175 ceph-mon[33399]: pgmap v1196: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:59.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:55:59 smithi131 ceph-mon[29369]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:59.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:55:59 smithi131 ceph-mon[40885]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:59.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:55:59 smithi175 ceph-mon[33399]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:01.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:01 smithi175 ceph-mon[33399]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:02.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:01 smithi131 ceph-mon[29369]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:02.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:01 smithi131 ceph-mon[40885]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:03.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:03.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:03.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:03 smithi175 ceph-mon[33399]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:04.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:03 smithi131 ceph-mon[29369]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:56:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:03] "GET /metrics HTTP/1.1" 200 226151 "" "Prometheus/2.43.0" 2024-09-10T18:56:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:03 smithi131 ceph-mon[40885]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:04.302 INFO:tasks.workunit.client.0.smithi131.stderr:+ pid=149417 2024-09-10T18:56:04.302 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_bytes 0 2024-09-10T18:56:04.302 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put two /etc/passwd 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.389+0000 7f8679e94640 1 -- 172.21.15.131:0/1320916842 >> v1:172.21.15.131:6789/0 conn(0x7f8674107d70 legacy=0x7f867410a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.389+0000 7f8679e94640 1 -- 172.21.15.131:0/1320916842 shutdown_connections 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.389+0000 7f8679e94640 1 -- 172.21.15.131:0/1320916842 >> 172.21.15.131:0/1320916842 conn(0x7f86740752f0 msgr2=0x7f86740f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.389+0000 7f8679e94640 1 -- 172.21.15.131:0/1320916842 shutdown_connections 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.389+0000 7f8679e94640 1 -- 172.21.15.131:0/1320916842 wait complete. 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8679e94640 1 Processor -- start 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8679e94640 1 -- start start 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8679e94640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f867411bfa0 con 0x7f8674107d70 2024-09-10T18:56:04.394 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8679e94640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f867411d160 con 0x7f8674104210 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8679e94640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f867411e360 con 0x7f86741014a0 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f86737fe640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f86741014a0 0x7f8674105300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:56116/0 (socket says 172.21.15.131:56116) 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8673fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f8674107d70 0x7f867411a6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58712/0 (socket says 172.21.15.131:58712) 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8673fff640 1 -- 172.21.15.131:0/3910686507 learned_addr learned my addr 172.21.15.131:0/3910686507 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966730023 0 0) 0x7f867411e360 con 0x7f86741014a0 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.390+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86600034e0 con 0x7f86741014a0 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446516290 0 0) 0x7f867411bfa0 con 0x7f8674107d70 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867411e360 con 0x7f8674107d70 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165442102 0 0) 0x7f867411d160 con 0x7f8674104210 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f867411bfa0 con 0x7f8674104210 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 174886411 0 0) 0x7f86600034e0 con 0x7f86741014a0 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f867411d160 con 0x7f86741014a0 2024-09-10T18:56:04.395 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f86640018c0 con 0x7f86741014a0 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3637787303 0 0) 0x7f867411e360 con 0x7f8674107d70 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f86600034e0 con 0x7f8674107d70 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f866c002420 con 0x7f8674107d70 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 984400020 0 0) 0x7f867411bfa0 con 0x7f8674104210 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f867411e360 con 0x7f8674104210 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f865c001620 con 0x7f8674104210 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2115779390 0 0) 0x7f867411d160 con 0x7f86741014a0 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 >> v1:172.21.15.175:6789/0 conn(0x7f8674104210 legacy=0x7f8674105a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 >> v1:172.21.15.131:6789/0 conn(0x7f8674107d70 legacy=0x7f867411a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867411f560 con 0x7f86741014a0 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867411e550 con 0x7f86741014a0 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f867411ea20 con 0x7f86741014a0 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f8664002410 con 0x7f86741014a0 2024-09-10T18:56:04.396 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.391+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f8664005a10 con 0x7f86741014a0 2024-09-10T18:56:04.397 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.392+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8654005010 con 0x7f86741014a0 2024-09-10T18:56:04.397 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.393+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f8664001fc0 con 0x7f86741014a0 2024-09-10T18:56:04.397 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.393+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(747..747 src has 1..747) v4 ==== 7810+0+0 (unknown 3890146330 0 0) 0x7f866408a520 con 0x7f86741014a0 2024-09-10T18:56:04.397 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.393+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=748}) v3 -- 0x7f867411d160 con 0x7f86741014a0 2024-09-10T18:56:04.399 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.397+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f8664056ba0 con 0x7f86741014a0 2024-09-10T18:56:04.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:56:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:56:04.555 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.554+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f86540052c0 con 0x7f86741014a0 2024-09-10T18:56:04.599 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.597+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 10 ==== osd_map(748..748 src has 1..748) v4 ==== 620+0+0 (unknown 200009135 0 0) 0x7f86640574d0 con 0x7f86741014a0 2024-09-10T18:56:04.599 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.597+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=749}) v3 -- 0x7f86600034e0 con 0x7f86741014a0 2024-09-10T18:56:04.603 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.602+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v748) v1 ==== 217+0+0 (unknown 1608740681 0 0) 0x7f8664056540 con 0x7f86741014a0 2024-09-10T18:56:04.697 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:04.696+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f8654001090 con 0x7f86741014a0 2024-09-10T18:56:04.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:04 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3910686507' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:04.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:04 smithi175 ceph-mon[33399]: from='client.57305 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:05.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:04 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3910686507' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:05.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:04 smithi131 ceph-mon[29369]: from='client.57305 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:04 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3910686507' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:05.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:04 smithi131 ceph-mon[40885]: from='client.57305 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:05.600 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.598+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 12 ==== osd_map(749..749 src has 1..749) v4 ==== 620+0+0 (unknown 2400037007 0 0) 0x7f8664056540 con 0x7f86741014a0 2024-09-10T18:56:05.600 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.598+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=750}) v3 -- 0x7f867411bfa0 con 0x7f86741014a0 2024-09-10T18:56:05.603 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.602+0000 7f8670ff9640 1 -- 172.21.15.131:0/3910686507 <== mon.2 v1:172.21.15.131:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v749) v1 ==== 217+0+0 (unknown 2394448519 0 0) 0x7f8664056720 con 0x7f86741014a0 2024-09-10T18:56:05.604 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_bytes = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:56:05.607 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.606+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 >> v1:172.21.15.131:6800/336444945 conn(0x7f866006fd70 legacy=0x7f8660072220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:05.608 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.606+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 >> v1:172.21.15.131:6790/0 conn(0x7f86741014a0 legacy=0x7f8674105300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:05.608 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.606+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 shutdown_connections 2024-09-10T18:56:05.608 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.606+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 >> 172.21.15.131:0/3910686507 conn(0x7f86740752f0 msgr2=0x7f867410b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:05.608 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.606+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 shutdown_connections 2024-09-10T18:56:05.608 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.606+0000 7f8679e94640 1 -- 172.21.15.131:0/3910686507 wait complete. 2024-09-10T18:56:05.618 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_objects 0 2024-09-10T18:56:05.701 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.697+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/3160019106 >> v1:172.21.15.131:6789/0 conn(0x7ff1ec1039a0 legacy=0x7ff1ec105d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/3160019106 shutdown_connections 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/3160019106 >> 172.21.15.131:0/3160019106 conn(0x7ff1ec074a80 msgr2=0x7ff1ec0f8cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/3160019106 shutdown_connections 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/3160019106 wait complete. 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 Processor -- start 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 -- start start 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.698+0000 7ff1f2f9d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1ec119d10 con 0x7ff1ec100bc0 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1f2f9d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1ec11af10 con 0x7ff1ec107500 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1f2f9d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1ec11c110 con 0x7ff1ec1039a0 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1ebfff640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7ff1ec1039a0 0x7ff1ec114b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:56138/0 (socket says 172.21.15.131:56138) 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1ebfff640 1 -- 172.21.15.131:0/605126904 learned_addr learned my addr 172.21.15.131:0/605126904 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:56:05.702 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1127988683 0 0) 0x7ff1ec11c110 con 0x7ff1ec1039a0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1dc0045f0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310262662 0 0) 0x7ff1ec119d10 con 0x7ff1ec100bc0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1ec11c110 con 0x7ff1ec100bc0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548416455 0 0) 0x7ff1ec11af10 con 0x7ff1ec107500 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1ec119d10 con 0x7ff1ec107500 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2180746748 0 0) 0x7ff1dc0045f0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1ec11af10 con 0x7ff1ec1039a0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff1d80020e0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3758695631 0 0) 0x7ff1ec11c110 con 0x7ff1ec100bc0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1dc0045f0 con 0x7ff1ec100bc0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff1e0001840 con 0x7ff1ec100bc0 2024-09-10T18:56:05.703 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3267241202 0 0) 0x7ff1ec119d10 con 0x7ff1ec107500 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1ec11c110 con 0x7ff1ec107500 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff1e4001c20 con 0x7ff1ec107500 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2653804408 0 0) 0x7ff1ec11af10 con 0x7ff1ec1039a0 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 >> v1:172.21.15.175:6789/0 conn(0x7ff1ec107500 legacy=0x7ff1ec118410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 >> v1:172.21.15.131:6789/0 conn(0x7ff1ec100bc0 legacy=0x7ff1ec0fff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec11d310 con 0x7ff1ec1039a0 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1ec119f00 con 0x7ff1ec1039a0 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1ec11a350 con 0x7ff1ec1039a0 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7ff1d8001af0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.699+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7ff1d8003660 con 0x7ff1ec1039a0 2024-09-10T18:56:05.704 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.700+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d0005010 con 0x7ff1ec1039a0 2024-09-10T18:56:05.705 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.701+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7ff1d80038f0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.705 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.701+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(749..749 src has 1..749) v4 ==== 7810+0+0 (unknown 1100516967 0 0) 0x7ff1d808a810 con 0x7ff1ec1039a0 2024-09-10T18:56:05.705 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.702+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=750}) v3 -- 0x7ff1ec11af10 con 0x7ff1ec1039a0 2024-09-10T18:56:05.707 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.706+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7ff1d8056ef0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.862 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:05.861+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7ff1d00052c0 con 0x7ff1ec1039a0 2024-09-10T18:56:05.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-mon[33399]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-mon[33399]: from='client.57305 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:56:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-mon[33399]: osdmap e748: 8 total, 8 up, 8 in 2024-09-10T18:56:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3910686507' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:05 smithi175 ceph-mon[33399]: from='client.57305 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:06.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[29369]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[29369]: from='client.57305 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[29369]: osdmap e748: 8 total, 8 up, 8 in 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3910686507' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[29369]: from='client.57305 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[40885]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[40885]: from='client.57305 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:56:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[40885]: osdmap e748: 8 total, 8 up, 8 in 2024-09-10T18:56:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3910686507' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:06.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:05 smithi131 ceph-mon[40885]: from='client.57305 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:56:06.611 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:06.610+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 10 ==== osd_map(750..750 src has 1..750) v4 ==== 620+0+0 (unknown 1865356698 0 0) 0x7ff1d8056c30 con 0x7ff1ec1039a0 2024-09-10T18:56:06.612 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:06.610+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=751}) v3 -- 0x7ff1dc0045f0 con 0x7ff1ec1039a0 2024-09-10T18:56:06.615 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:06.614+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v750) v1 ==== 221+0+0 (unknown 827664392 0 0) 0x7ff1d8056890 con 0x7ff1ec1039a0 2024-09-10T18:56:06.712 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:06.711+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7ff1d0000be0 con 0x7ff1ec1039a0 2024-09-10T18:56:06.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:06 smithi175 ceph-mon[33399]: from='client.57305 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:56:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:06 smithi175 ceph-mon[33399]: osdmap e749: 8 total, 8 up, 8 in 2024-09-10T18:56:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:06 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/605126904' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:06.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:06 smithi175 ceph-mon[33399]: from='client.57311 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[29369]: from='client.57305 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[29369]: osdmap e749: 8 total, 8 up, 8 in 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/605126904' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[29369]: from='client.57311 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[40885]: from='client.57305 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[40885]: osdmap e749: 8 total, 8 up, 8 in 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/605126904' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:07.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:06 smithi131 ceph-mon[40885]: from='client.57311 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:07.211 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.209+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 12 ==== osd_map(751..751 src has 1..751) v4 ==== 620+0+0 (unknown 2771580380 0 0) 0x7ff1d8057820 con 0x7ff1ec1039a0 2024-09-10T18:56:07.211 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.209+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=752}) v3 -- 0x7ff1ec119d10 con 0x7ff1ec1039a0 2024-09-10T18:56:07.214 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.213+0000 7ff1e97fa640 1 -- 172.21.15.131:0/605126904 <== mon.2 v1:172.21.15.131:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v751) v1 ==== 221+0+0 (unknown 2969958889 0 0) 0x7ff1d8056a70 con 0x7ff1ec1039a0 2024-09-10T18:56:07.215 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:56:07.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.216+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 >> v1:172.21.15.131:6800/336444945 conn(0x7ff1dc070ed0 legacy=0x7ff1dc073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:07.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.216+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 >> v1:172.21.15.131:6790/0 conn(0x7ff1ec1039a0 legacy=0x7ff1ec114b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:07.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.217+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 shutdown_connections 2024-09-10T18:56:07.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.217+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 >> 172.21.15.131:0/605126904 conn(0x7ff1ec074a80 msgr2=0x7ff1ec0fac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:07.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.217+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 shutdown_connections 2024-09-10T18:56:07.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.217+0000 7ff1f2f9d640 1 -- 172.21.15.131:0/605126904 wait complete. 2024-09-10T18:56:07.227 INFO:tasks.workunit.client.0.smithi131.stderr:+ wait 149417 2024-09-10T18:56:07.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ '[' 0 -ne 0 ']' 2024-09-10T18:56:07.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ true 2024-09-10T18:56:07.228 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put three /etc/passwd 2024-09-10T18:56:07.257 INFO:tasks.workunit.client.0.smithi131.stderr:++ uuidgen 2024-09-10T18:56:07.258 INFO:tasks.workunit.client.0.smithi131.stderr:+ pp=949e8d97-c4e4-4f93-ad36-26b7449a9224 2024-09-10T18:56:07.258 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool create 949e8d97-c4e4-4f93-ad36-26b7449a9224 12 2024-09-10T18:56:07.337 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.333+0000 7f7908046640 1 -- 172.21.15.131:0/3778170417 >> v1:172.21.15.131:6790/0 conn(0x7f7900108ec0 legacy=0x7f790010b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:07.337 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.333+0000 7f7908046640 1 -- 172.21.15.131:0/3778170417 shutdown_connections 2024-09-10T18:56:07.337 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.333+0000 7f7908046640 1 -- 172.21.15.131:0/3778170417 >> 172.21.15.131:0/3778170417 conn(0x7f7900074a40 msgr2=0x7f79000f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:07.337 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.333+0000 7f7908046640 1 -- 172.21.15.131:0/3778170417 shutdown_connections 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.333+0000 7f7908046640 1 -- 172.21.15.131:0/3778170417 wait complete. 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7908046640 1 Processor -- start 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7908046640 1 -- start start 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7908046640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f790011bf80 con 0x7f7900105360 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7908046640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f790011d140 con 0x7f7900108ec0 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7908046640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f790011e340 con 0x7f79001025f0 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7905de2640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f79001025f0 0x7f79000fe5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:56168/0 (socket says 172.21.15.131:56168) 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f79055e1640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f7900105360 0x7f79000fecf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58778/0 (socket says 172.21.15.131:58778) 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f7905de2640 1 -- 172.21.15.131:0/2577378663 learned_addr learned my addr 172.21.15.131:0/2577378663 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361360418 0 0) 0x7f790011bf80 con 0x7f7900105360 2024-09-10T18:56:07.338 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78ec003200 con 0x7f7900105360 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3246020612 0 0) 0x7f790011e340 con 0x7f79001025f0 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790011bf80 con 0x7f79001025f0 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646627636 0 0) 0x7f790011d140 con 0x7f7900108ec0 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f790011e340 con 0x7f7900108ec0 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 229733312 0 0) 0x7f78ec003200 con 0x7f7900105360 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f790011d140 con 0x7f7900105360 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.334+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78f0001a30 con 0x7f7900105360 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 426840642 0 0) 0x7f790011bf80 con 0x7f79001025f0 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78ec003200 con 0x7f79001025f0 2024-09-10T18:56:07.339 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78fc001880 con 0x7f79001025f0 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 555041 0 0) 0x7f790011e340 con 0x7f7900108ec0 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f790011bf80 con 0x7f7900108ec0 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78f8001740 con 0x7f7900108ec0 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3382387362 0 0) 0x7f790011d140 con 0x7f7900105360 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 >> v1:172.21.15.131:6790/0 conn(0x7f79001025f0 legacy=0x7f79000fe5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 >> v1:172.21.15.175:6789/0 conn(0x7f7900108ec0 legacy=0x7f790011a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f790011f540 con 0x7f7900105360 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790011c170 con 0x7f7900105360 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f790011c640 con 0x7f7900105360 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f78f0002350 con 0x7f7900105360 2024-09-10T18:56:07.340 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.335+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f78f0005990 con 0x7f7900105360 2024-09-10T18:56:07.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.336+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e4005010 con 0x7f7900105360 2024-09-10T18:56:07.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.336+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f78f00024c0 con 0x7f7900105360 2024-09-10T18:56:07.341 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.337+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(751..751 src has 1..751) v4 ==== 7810+0+0 (unknown 598036556 0 0) 0x7f78f008a450 con 0x7f7900105360 2024-09-10T18:56:07.342 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.341+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f78f0056690 con 0x7f7900105360 2024-09-10T18:56:07.497 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:07.496+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12} v 0) v1 -- 0x7f78e4005840 con 0x7f7900105360 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: from='client.57311 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: osdmap e750: 8 total, 8 up, 8 in 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/605126904' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: from='client.57311 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: from='client.57311 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[29369]: osdmap e751: 8 total, 8 up, 8 in 2024-09-10T18:56:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: from='client.57311 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: osdmap e750: 8 total, 8 up, 8 in 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/605126904' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: from='client.57311 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: from='client.57311 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:56:08.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:07 smithi131 ceph-mon[40885]: osdmap e751: 8 total, 8 up, 8 in 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: from='client.57311 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: osdmap e750: 8 total, 8 up, 8 in 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/605126904' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: from='client.57311 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: pool 'e7116ce6-52bb-4120-b144-3681a246036c' no longer out of quota; removing NO_QUOTA flag 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: from='client.57311 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:56:08.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:07 smithi175 ceph-mon[33399]: osdmap e751: 8 total, 8 up, 8 in 2024-09-10T18:56:08.210 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.209+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]=0 pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' created v752) v1 ==== 176+0+0 (unknown 3855342099 0 0) 0x7f78f00564b0 con 0x7f7900105360 2024-09-10T18:56:08.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.308+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12} v 0) v1 -- 0x7f78e40052c0 con 0x7f7900105360 2024-09-10T18:56:08.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.309+0000 7f78f67fc640 1 -- 172.21.15.131:0/2577378663 <== mon.0 v1:172.21.15.131:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]=0 pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' already exists v752) v1 ==== 183+0+0 (unknown 934893112 0 0) 0x7f78f00564b0 con 0x7f7900105360 2024-09-10T18:56:08.310 INFO:tasks.workunit.client.0.smithi131.stderr:pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' already exists 2024-09-10T18:56:08.312 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.311+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 >> v1:172.21.15.131:6800/336444945 conn(0x7f78ec06fd80 legacy=0x7f78ec072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:08.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.311+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 >> v1:172.21.15.131:6789/0 conn(0x7f7900105360 legacy=0x7f79000fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:08.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.311+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 shutdown_connections 2024-09-10T18:56:08.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.311+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 >> 172.21.15.131:0/2577378663 conn(0x7f7900074a40 msgr2=0x7f79000f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:08.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.311+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 shutdown_connections 2024-09-10T18:56:08.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.311+0000 7f7908046640 1 -- 172.21.15.131:0/2577378663 wait complete. 2024-09-10T18:56:08.321 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool application enable 949e8d97-c4e4-4f93-ad36-26b7449a9224 rados 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 -- 172.21.15.131:0/1571058317 >> v1:172.21.15.131:6789/0 conn(0x7f64a8108e90 legacy=0x7f64a810b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 -- 172.21.15.131:0/1571058317 shutdown_connections 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 -- 172.21.15.131:0/1571058317 >> 172.21.15.131:0/1571058317 conn(0x7f64a80f8100 msgr2=0x7f64a80fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 -- 172.21.15.131:0/1571058317 shutdown_connections 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 -- 172.21.15.131:0/1571058317 wait complete. 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 Processor -- start 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.396+0000 7f64b0100640 1 -- start start 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64b0100640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64a811bf50 con 0x7f64a8102540 2024-09-10T18:56:08.400 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64b0100640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64a811d110 con 0x7f64a8105300 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64b0100640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64a811e310 con 0x7f64a8108e90 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64ade9c640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f64a8102540 0x7f64a80fe5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:58796/0 (socket says 172.21.15.131:58796) 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64ade9c640 1 -- 172.21.15.131:0/3504975392 learned_addr learned my addr 172.21.15.131:0/3504975392 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106916973 0 0) 0x7f64a811e310 con 0x7f64a8108e90 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64940034e0 con 0x7f64a8108e90 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228152509 0 0) 0x7f64a811bf50 con 0x7f64a8102540 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a811e310 con 0x7f64a8102540 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2579225500 0 0) 0x7f64a811d110 con 0x7f64a8105300 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64a811bf50 con 0x7f64a8105300 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3470444646 0 0) 0x7f64940034e0 con 0x7f64a8108e90 2024-09-10T18:56:08.401 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64a811d110 con 0x7f64a8108e90 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f64a0002da0 con 0x7f64a8108e90 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3657034675 0 0) 0x7f64a811e310 con 0x7f64a8102540 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64940034e0 con 0x7f64a8102540 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f64a4001830 con 0x7f64a8102540 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3143780362 0 0) 0x7f64a811bf50 con 0x7f64a8105300 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64a811e310 con 0x7f64a8105300 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f64980018c0 con 0x7f64a8105300 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 366692745 0 0) 0x7f64a811d110 con 0x7f64a8108e90 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 >> v1:172.21.15.175:6789/0 conn(0x7f64a8105300 legacy=0x7f64a80fecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 >> v1:172.21.15.131:6789/0 conn(0x7f64a8102540 legacy=0x7f64a80fe5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64a811f510 con 0x7f64a8108e90 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64a811c140 con 0x7f64a8108e90 2024-09-10T18:56:08.402 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.397+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a811c590 con 0x7f64a8108e90 2024-09-10T18:56:08.403 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.398+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f64a0001e30 con 0x7f64a8108e90 2024-09-10T18:56:08.403 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.398+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f64a0003960 con 0x7f64a8108e90 2024-09-10T18:56:08.403 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.398+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648c005040 con 0x7f64a8108e90 2024-09-10T18:56:08.403 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.399+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f64a00024e0 con 0x7f64a8108e90 2024-09-10T18:56:08.403 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.400+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(752..752 src has 1..752) v4 ==== 8185+0+0 (unknown 1069242919 0 0) 0x7f64a0088310 con 0x7f64a8108e90 2024-09-10T18:56:08.405 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.404+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f64a0054870 con 0x7f64a8108e90 2024-09-10T18:56:08.561 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:08.560+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"} v 0) v1 -- 0x7f648c0052f0 con 0x7f64a8108e90 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]: dispatch 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]': finished 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[29369]: osdmap e752: 8 total, 8 up, 8 in 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]: dispatch 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3504975392' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[29369]: from='client.57332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:09.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]: dispatch 2024-09-10T18:56:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]': finished 2024-09-10T18:56:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[40885]: osdmap e752: 8 total, 8 up, 8 in 2024-09-10T18:56:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]: dispatch 2024-09-10T18:56:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3504975392' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:09.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:08 smithi131 ceph-mon[40885]: from='client.57332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:09.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]: dispatch 2024-09-10T18:56:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]': finished 2024-09-10T18:56:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:08 smithi175 ceph-mon[33399]: osdmap e752: 8 total, 8 up, 8 in 2024-09-10T18:56:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2577378663' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pg_num": 12}]: dispatch 2024-09-10T18:56:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:08 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3504975392' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:09.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:08 smithi175 ceph-mon[33399]: from='client.57332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:09.217 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:09.216+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]=0 enabled application 'rados' on pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' v753) v1 ==== 213+0+0 (unknown 1716464088 0 0) 0x7f64a0001550 con 0x7f64a8108e90 2024-09-10T18:56:09.317 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:09.316+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"} v 0) v1 -- 0x7f648c005a30 con 0x7f64a8108e90 2024-09-10T18:56:10.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[29369]: pgmap v1207: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-10T18:56:10.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[29369]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[29369]: from='client.57332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]': finished 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[29369]: osdmap e753: 8 total, 8 up, 8 in 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/3504975392' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[29369]: from='client.57332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[40885]: pgmap v1207: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[40885]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[40885]: from='client.57332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]': finished 2024-09-10T18:56:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[40885]: osdmap e753: 8 total, 8 up, 8 in 2024-09-10T18:56:10.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/3504975392' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:10.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:09 smithi131 ceph-mon[40885]: from='client.57332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:09 smithi175 ceph-mon[33399]: pgmap v1207: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-10T18:56:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:09 smithi175 ceph-mon[33399]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:09 smithi175 ceph-mon[33399]: from='client.57332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]': finished 2024-09-10T18:56:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:09 smithi175 ceph-mon[33399]: osdmap e753: 8 total, 8 up, 8 in 2024-09-10T18:56:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:09 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/3504975392' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:10.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:09 smithi175 ceph-mon[33399]: from='client.57332 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]: dispatch 2024-09-10T18:56:10.218 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.216+0000 7f649e7fc640 1 -- 172.21.15.131:0/3504975392 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]=0 enabled application 'rados' on pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' v754) v1 ==== 213+0+0 (unknown 674050802 0 0) 0x7f64a0054210 con 0x7f64a8108e90 2024-09-10T18:56:10.218 INFO:tasks.workunit.client.0.smithi131.stderr:enabled application 'rados' on pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' 2024-09-10T18:56:10.221 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.219+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 >> v1:172.21.15.131:6800/336444945 conn(0x7f649406fea0 legacy=0x7f6494072350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:10.221 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.219+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 >> v1:172.21.15.131:6790/0 conn(0x7f64a8108e90 legacy=0x7f64a811a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:10.221 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.219+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 shutdown_connections 2024-09-10T18:56:10.221 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.219+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 >> 172.21.15.131:0/3504975392 conn(0x7f64a80f8100 msgr2=0x7f64a810baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:10.221 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.219+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 shutdown_connections 2024-09-10T18:56:10.221 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.219+0000 7f64b0100640 1 -- 172.21.15.131:0/3504975392 wait complete. 2024-09-10T18:56:10.230 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota 949e8d97-c4e4-4f93-ad36-26b7449a9224 max_objects 10 2024-09-10T18:56:10.309 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.307+0000 7f150aaef640 1 -- 172.21.15.131:0/4095623284 >> v1:172.21.15.131:6790/0 conn(0x7f1504108e80 legacy=0x7f150410b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:10.309 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.307+0000 7f150aaef640 1 -- 172.21.15.131:0/4095623284 shutdown_connections 2024-09-10T18:56:10.309 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.307+0000 7f150aaef640 1 -- 172.21.15.131:0/4095623284 >> 172.21.15.131:0/4095623284 conn(0x7f15040f8130 msgr2=0x7f15040fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:10.309 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.307+0000 7f150aaef640 1 -- 172.21.15.131:0/4095623284 shutdown_connections 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.307+0000 7f150aaef640 1 -- 172.21.15.131:0/4095623284 wait complete. 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f150aaef640 1 Processor -- start 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f150aaef640 1 -- start start 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f150aaef640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f15040ff330 con 0x7f15041052f0 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f150aaef640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f150411d380 con 0x7f1504102530 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f150aaef640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f150411e520 con 0x7f1504108e80 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f1503fff640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f15041052f0 0x7f1504117520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55420/0 (socket says 172.21.15.131:55420) 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f150908c640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f1504108e80 0x7f150411ac50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:34222/0 (socket says 172.21.15.131:34222) 2024-09-10T18:56:10.310 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f1503fff640 1 -- 172.21.15.131:0/4078789692 learned_addr learned my addr 172.21.15.131:0/4078789692 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:56:10.312 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3712555249 0 0) 0x7f150411e520 con 0x7f1504108e80 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f40034e0 con 0x7f1504108e80 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2164824259 0 0) 0x7f15040ff330 con 0x7f15041052f0 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f150411e520 con 0x7f15041052f0 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431296416 0 0) 0x7f150411d380 con 0x7f1504102530 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.308+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15040ff330 con 0x7f1504102530 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 207035579 0 0) 0x7f14f40034e0 con 0x7f1504108e80 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f150411d380 con 0x7f1504108e80 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f14fc003010 con 0x7f1504108e80 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1665531301 0 0) 0x7f150411e520 con 0x7f15041052f0 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14f40034e0 con 0x7f15041052f0 2024-09-10T18:56:10.313 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f14f00017c0 con 0x7f15041052f0 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1704079376 0 0) 0x7f15040ff330 con 0x7f1504102530 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f150411e520 con 0x7f1504102530 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f14f8001840 con 0x7f1504102530 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1871368954 0 0) 0x7f150411d380 con 0x7f1504108e80 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 >> v1:172.21.15.175:6789/0 conn(0x7f1504102530 legacy=0x7f15040fe6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 >> v1:172.21.15.131:6789/0 conn(0x7f15041052f0 legacy=0x7f1504117520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150411f720 con 0x7f1504108e80 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150411c3d0 con 0x7f1504108e80 2024-09-10T18:56:10.314 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f150411c820 con 0x7f1504108e80 2024-09-10T18:56:10.315 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f14fc002760 con 0x7f1504108e80 2024-09-10T18:56:10.315 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.309+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f14fc0038d0 con 0x7f1504108e80 2024-09-10T18:56:10.315 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.310+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e8005010 con 0x7f1504108e80 2024-09-10T18:56:10.315 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.310+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f14fc002310 con 0x7f1504108e80 2024-09-10T18:56:10.315 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.311+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(754..754 src has 1..754) v4 ==== 8198+0+0 (unknown 2216855518 0 0) 0x7f14fc08ac50 con 0x7f1504108e80 2024-09-10T18:56:10.316 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.315+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f14fc057170 con 0x7f1504108e80 2024-09-10T18:56:10.473 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:10.472+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f14e80052c0 con 0x7f1504108e80 2024-09-10T18:56:11.225 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:11.224+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 949e8d97-c4e4-4f93-ad36-26b7449a9224 v755) v1 ==== 223+0+0 (unknown 2637187188 0 0) 0x7f14fc0025c0 con 0x7f1504108e80 2024-09-10T18:56:11.321 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:11.320+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f14e8001090 con 0x7f1504108e80 2024-09-10T18:56:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[29369]: from='client.57332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]': finished 2024-09-10T18:56:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[29369]: osdmap e754: 8 total, 8 up, 8 in 2024-09-10T18:56:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4078789692' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[29369]: from='client.57338 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:11.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[29369]: pgmap v1210: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-10T18:56:11.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[40885]: from='client.57332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]': finished 2024-09-10T18:56:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[40885]: osdmap e754: 8 total, 8 up, 8 in 2024-09-10T18:56:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4078789692' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[40885]: from='client.57338 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:11.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:11 smithi131 ceph-mon[40885]: pgmap v1210: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-10T18:56:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:11 smithi175 ceph-mon[33399]: from='client.57332 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "app": "rados"}]': finished 2024-09-10T18:56:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:11 smithi175 ceph-mon[33399]: osdmap e754: 8 total, 8 up, 8 in 2024-09-10T18:56:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:11 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4078789692' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:11 smithi175 ceph-mon[33399]: from='client.57338 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:11.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:11 smithi175 ceph-mon[33399]: pgmap v1210: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.7 KiB/s wr, 1 op/s 2024-09-10T18:56:12.227 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.225+0000 7f15017fa640 1 -- 172.21.15.131:0/4078789692 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 949e8d97-c4e4-4f93-ad36-26b7449a9224 v756) v1 ==== 223+0+0 (unknown 1711555797 0 0) 0x7f14fc056b10 con 0x7f1504108e80 2024-09-10T18:56:12.227 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 10 for pool 949e8d97-c4e4-4f93-ad36-26b7449a9224 2024-09-10T18:56:12.231 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.229+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 >> v1:172.21.15.131:6800/336444945 conn(0x7f14f406fe60 legacy=0x7f14f4072310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:12.231 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.229+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 >> v1:172.21.15.131:6790/0 conn(0x7f1504108e80 legacy=0x7f150411ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:56:12.231 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.230+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 shutdown_connections 2024-09-10T18:56:12.231 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.230+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 >> 172.21.15.131:0/4078789692 conn(0x7f15040f8130 msgr2=0x7f15040f8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:56:12.231 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.230+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 shutdown_connections 2024-09-10T18:56:12.231 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:56:12.230+0000 7f150aaef640 1 -- 172.21.15.131:0/4078789692 wait complete. 2024-09-10T18:56:12.242 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[29369]: from='client.57338 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[29369]: osdmap e755: 8 total, 8 up, 8 in 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/4078789692' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[29369]: from='client.57338 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[40885]: from='client.57338 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[40885]: osdmap e755: 8 total, 8 up, 8 in 2024-09-10T18:56:12.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/4078789692' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:12.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:12 smithi131 ceph-mon[40885]: from='client.57338 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:12 smithi175 ceph-mon[33399]: from='client.57338 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:56:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:12 smithi175 ceph-mon[33399]: osdmap e755: 8 total, 8 up, 8 in 2024-09-10T18:56:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:12 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/4078789692' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:12.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:12 smithi175 ceph-mon[33399]: from='client.57338 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T18:56:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-mon[29369]: from='client.57338 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:56:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-mon[29369]: osdmap e756: 8 total, 8 up, 8 in 2024-09-10T18:56:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-mon[29369]: pgmap v1213: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:56:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-mon[40885]: from='client.57338 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:56:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-mon[40885]: osdmap e756: 8 total, 8 up, 8 in 2024-09-10T18:56:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-mon[40885]: pgmap v1213: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:56:13.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:13 smithi175 ceph-mon[33399]: from='client.57338 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "field": "max_objects", "val": "10"}]': finished 2024-09-10T18:56:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:13 smithi175 ceph-mon[33399]: osdmap e756: 8 total, 8 up, 8 in 2024-09-10T18:56:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:13 smithi175 ceph-mon[33399]: pgmap v1213: 188 pgs: 10 creating+peering, 2 unknown, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:56:14.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:56:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:13] "GET /metrics HTTP/1.1" 200 226151 "" "Prometheus/2.43.0" 2024-09-10T18:56:14.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:56:14 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:56:15.328 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:56:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:15.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:15 smithi175 ceph-mon[33399]: pgmap v1214: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 968 B/s rd, 0 op/s 2024-09-10T18:56:15.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:15.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:15 smithi131 ceph-mon[29369]: pgmap v1214: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 968 B/s rd, 0 op/s 2024-09-10T18:56:15.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:15.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:15 smithi131 ceph-mon[40885]: pgmap v1214: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 968 B/s rd, 0 op/s 2024-09-10T18:56:17.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[29369]: pgmap v1215: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 814 B/s rd, 0 op/s 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[29369]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[40885]: pgmap v1215: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 814 B/s rd, 0 op/s 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[40885]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:17.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:17 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:17 smithi175 ceph-mon[33399]: pgmap v1215: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 814 B/s rd, 0 op/s 2024-09-10T18:56:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:17 smithi175 ceph-mon[33399]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:17.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:17 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:19.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:19 smithi175 ceph-mon[33399]: pgmap v1216: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:19.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:19 smithi131 ceph-mon[40885]: pgmap v1216: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:19.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:19 smithi131 ceph-mon[29369]: pgmap v1216: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:21.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:21 smithi131 ceph-mon[29369]: pgmap v1217: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:56:21.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:21 smithi131 ceph-mon[40885]: pgmap v1217: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:56:21.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:21 smithi175 ceph-mon[33399]: pgmap v1217: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:56:23.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:23 smithi131 ceph-mon[29369]: pgmap v1218: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 996 B/s rd, 0 op/s 2024-09-10T18:56:23.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:23 smithi131 ceph-mon[40885]: pgmap v1218: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 996 B/s rd, 0 op/s 2024-09-10T18:56:23.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:56:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:23] "GET /metrics HTTP/1.1" 200 228641 "" "Prometheus/2.43.0" 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:23 smithi175 ceph-mon[33399]: pgmap v1218: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 996 B/s rd, 0 op/s 2024-09-10T18:56:24.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:56:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:56:25.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:56:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:25.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:25.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:25 smithi131 ceph-mon[29369]: pgmap v1219: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:25.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:25 smithi131 ceph-mon[40885]: pgmap v1219: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:25.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:25 smithi175 ceph-mon[33399]: pgmap v1219: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:27.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:27 smithi131 ceph-mon[29369]: pgmap v1220: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:27.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:27 smithi131 ceph-mon[40885]: pgmap v1220: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:27.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:27 smithi175 ceph-mon[33399]: pgmap v1220: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:29.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:29 smithi131 ceph-mon[29369]: pgmap v1221: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:29.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:29 smithi131 ceph-mon[40885]: pgmap v1221: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:29.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:29 smithi175 ceph-mon[33399]: pgmap v1221: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:31.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:31 smithi175 ceph-mon[33399]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:32.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:31 smithi131 ceph-mon[29369]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:32.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:31 smithi131 ceph-mon[40885]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:56:32.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:33.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:33.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:33 smithi175 ceph-mon[33399]: pgmap v1223: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:34.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:33 smithi131 ceph-mon[29369]: pgmap v1223: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:34.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:33 smithi131 ceph-mon[40885]: pgmap v1223: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:56:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:33] "GET /metrics HTTP/1.1" 200 228599 "" "Prometheus/2.43.0" 2024-09-10T18:56:34.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:56:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:56:35.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:56:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:35.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:35 smithi175 ceph-mon[33399]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:36.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:35 smithi131 ceph-mon[29369]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:36.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:36.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:35 smithi131 ceph-mon[40885]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:37.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:37 smithi175 ceph-mon[33399]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:38.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:37 smithi131 ceph-mon[29369]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:37 smithi131 ceph-mon[40885]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:39.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:39 smithi175 ceph-mon[33399]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:40.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:39 smithi131 ceph-mon[29369]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:40.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:39 smithi131 ceph-mon[40885]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:41.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:41 smithi175 ceph-mon[33399]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:41.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:42.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[29369]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[40885]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:42.243 INFO:tasks.workunit.client.0.smithi131.stderr:++ seq 1 10 2024-09-10T18:56:42.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.244 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj1 /etc/passwd 2024-09-10T18:56:42.276 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.276 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj2 /etc/passwd 2024-09-10T18:56:42.305 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.305 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj3 /etc/passwd 2024-09-10T18:56:42.334 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.336 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj4 /etc/passwd 2024-09-10T18:56:42.363 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.363 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj5 /etc/passwd 2024-09-10T18:56:42.391 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.391 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj6 /etc/passwd 2024-09-10T18:56:42.419 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.419 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj7 /etc/passwd 2024-09-10T18:56:42.448 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.448 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj8 /etc/passwd 2024-09-10T18:56:42.476 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.476 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj9 /etc/passwd 2024-09-10T18:56:42.504 INFO:tasks.workunit.client.0.smithi131.stderr:+ for f in `seq 1 10` 2024-09-10T18:56:42.505 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p 949e8d97-c4e4-4f93-ad36-26b7449a9224 put obj10 /etc/passwd 2024-09-10T18:56:42.533 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-09-10T18:56:43.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:43 smithi175 ceph-mon[33399]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:44.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:43 smithi131 ceph-mon[29369]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:56:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:43] "GET /metrics HTTP/1.1" 200 228599 "" "Prometheus/2.43.0" 2024-09-10T18:56:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:43 smithi131 ceph-mon[40885]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:56:44.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:56:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:56:45.328 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:56:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:45.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:45.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:45 smithi175 ceph-mon[33399]: pgmap v1229: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T18:56:46.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:45 smithi131 ceph-mon[29369]: pgmap v1229: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T18:56:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:45 smithi131 ceph-mon[40885]: pgmap v1229: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T18:56:47.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:47 smithi175 ceph-mon[33399]: pgmap v1230: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:56:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:47 smithi175 ceph-mon[33399]: pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' is full (reached quota's max_objects: 10) 2024-09-10T18:56:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:47 smithi175 ceph-mon[33399]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:56:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:47 smithi175 ceph-mon[33399]: osdmap e757: 8 total, 8 up, 8 in 2024-09-10T18:56:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:47.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:48.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[29369]: pgmap v1230: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[29369]: pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' is full (reached quota's max_objects: 10) 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[29369]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[29369]: osdmap e757: 8 total, 8 up, 8 in 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[40885]: pgmap v1230: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 11 KiB/s wr, 1 op/s 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[40885]: pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' is full (reached quota's max_objects: 10) 2024-09-10T18:56:48.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[40885]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:56:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[40885]: osdmap e757: 8 total, 8 up, 8 in 2024-09-10T18:56:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:56:48.078 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:49.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:49 smithi175 ceph-mon[33399]: pgmap v1232: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:50.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:49 smithi131 ceph-mon[29369]: pgmap v1232: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:50.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:49 smithi131 ceph-mon[40885]: pgmap v1232: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:51.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:51 smithi175 ceph-mon[33399]: pgmap v1233: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:52.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:51 smithi131 ceph-mon[29369]: pgmap v1233: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:52.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:51 smithi131 ceph-mon[40885]: pgmap v1233: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:53.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:53 smithi175 ceph-mon[33399]: pgmap v1234: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:54.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:53 smithi131 ceph-mon[29369]: pgmap v1234: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:54.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:56:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:53] "GET /metrics HTTP/1.1" 200 228596 "" "Prometheus/2.43.0" 2024-09-10T18:56:54.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:53 smithi131 ceph-mon[40885]: pgmap v1234: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T18:56:54.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:56:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:56:55.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:56:55 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:56:55] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:55 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:55.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:55 smithi175 ceph-mon[33399]: pgmap v1235: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:56:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:55 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:56.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:55 smithi131 ceph-mon[29369]: pgmap v1235: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:56:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:55 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:56.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:55 smithi131 ceph-mon[40885]: pgmap v1235: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:56:57.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:57 smithi175 ceph-mon[33399]: pgmap v1236: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:56:58.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:57 smithi131 ceph-mon[29369]: pgmap v1236: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:56:58.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:57 smithi131 ceph-mon[40885]: pgmap v1236: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:56:59.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:56:59 smithi175 ceph-mon[33399]: pgmap v1237: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:57:00.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:56:59 smithi131 ceph-mon[29369]: pgmap v1237: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:57:00.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:56:59 smithi131 ceph-mon[40885]: pgmap v1237: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:57:01.879 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:01 smithi131 ceph-mon[29369]: pgmap v1238: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:01.879 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:01 smithi131 ceph-mon[40885]: pgmap v1238: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:01.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:01 smithi175 ceph-mon[33399]: pgmap v1238: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:02.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:02 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:03.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:02 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:03.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:02 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:03.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:03 smithi175 ceph-mon[33399]: pgmap v1239: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:04.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:03 smithi131 ceph-mon[29369]: pgmap v1239: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:04.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:03 smithi131 ceph-mon[40885]: pgmap v1239: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:04.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:03] "GET /metrics HTTP/1.1" 200 228658 "" "Prometheus/2.43.0" 2024-09-10T18:57:04.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:57:04 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:57:05.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:05 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:05] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:05 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:05.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:05 smithi175 ceph-mon[33399]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:06.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:05 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:06.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:05 smithi131 ceph-mon[29369]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:05 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:06.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:05 smithi131 ceph-mon[40885]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:07.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:07 smithi175 ceph-mon[33399]: pgmap v1241: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:08.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:07 smithi131 ceph-mon[29369]: pgmap v1241: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:08.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:07 smithi131 ceph-mon[40885]: pgmap v1241: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:09.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:09 smithi175 ceph-mon[33399]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:10.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:09 smithi131 ceph-mon[29369]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:10.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:09 smithi131 ceph-mon[40885]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:11.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:11 smithi175 ceph-mon[33399]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:12.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:11 smithi131 ceph-mon[29369]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:12.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:11 smithi131 ceph-mon[40885]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:12.534 INFO:tasks.workunit.client.0.smithi131.stderr:+ rados -p e7116ce6-52bb-4120-b144-3681a246036c put threemore /etc/passwd 2024-09-10T18:57:12.567 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_bytes 0 2024-09-10T18:57:12.653 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.648+0000 7f11454ef640 1 -- 172.21.15.131:0/2193508828 >> v1:172.21.15.131:6790/0 conn(0x7f11401052f0 legacy=0x7f11401076d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:12.654 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.648+0000 7f11454ef640 1 -- 172.21.15.131:0/2193508828 shutdown_connections 2024-09-10T18:57:12.654 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.648+0000 7f11454ef640 1 -- 172.21.15.131:0/2193508828 >> 172.21.15.131:0/2193508828 conn(0x7f11400f8130 msgr2=0x7f11400fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:12.654 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.648+0000 7f11454ef640 1 -- 172.21.15.131:0/2193508828 shutdown_connections 2024-09-10T18:57:12.654 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.648+0000 7f11454ef640 1 -- 172.21.15.131:0/2193508828 wait complete. 2024-09-10T18:57:12.654 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f11454ef640 1 Processor -- start 2024-09-10T18:57:12.654 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f11454ef640 1 -- start start 2024-09-10T18:57:12.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f11454ef640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1140101ad0 con 0x7f1140108e80 2024-09-10T18:57:12.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f11454ef640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1140101c60 con 0x7f1140102530 2024-09-10T18:57:12.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f11454ef640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1140101e50 con 0x7f11401052f0 2024-09-10T18:57:12.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f113e7fc640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f11401052f0 0x7f11401198a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:51282/0 (socket says 172.21.15.131:51282) 2024-09-10T18:57:12.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f113e7fc640 1 -- 172.21.15.131:0/1725276334 learned_addr learned my addr 172.21.15.131:0/1725276334 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:57:12.655 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.649+0000 7f113f7fe640 1 --1- 172.21.15.131:0/1725276334 >> v1:172.21.15.131:6789/0 conn(0x7f1140108e80 0x7f114011cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55296/0 (socket says 172.21.15.131:55296) 2024-09-10T18:57:12.656 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128228012 0 0) 0x7f1140101e50 con 0x7f11401052f0 2024-09-10T18:57:12.656 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1128003200 con 0x7f11401052f0 2024-09-10T18:57:12.656 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031490415 0 0) 0x7f1140101ad0 con 0x7f1140108e80 2024-09-10T18:57:12.656 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1140101e50 con 0x7f1140108e80 2024-09-10T18:57:12.656 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475996618 0 0) 0x7f1140101c60 con 0x7f1140102530 2024-09-10T18:57:12.657 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1140101ad0 con 0x7f1140102530 2024-09-10T18:57:12.657 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1396807764 0 0) 0x7f1140101e50 con 0x7f1140108e80 2024-09-10T18:57:12.657 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1140101c60 con 0x7f1140108e80 2024-09-10T18:57:12.657 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3090636670 0 0) 0x7f1128003200 con 0x7f11401052f0 2024-09-10T18:57:12.657 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1140101e50 con 0x7f11401052f0 2024-09-10T18:57:12.657 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f1138001b80 con 0x7f1140108e80 2024-09-10T18:57:12.658 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f11340015d0 con 0x7f11401052f0 2024-09-10T18:57:12.658 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2549890809 0 0) 0x7f1140101ad0 con 0x7f1140102530 2024-09-10T18:57:12.658 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1128003200 con 0x7f1140102530 2024-09-10T18:57:12.658 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f11300018d0 con 0x7f1140102530 2024-09-10T18:57:12.658 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 561006575 0 0) 0x7f1140101c60 con 0x7f1140108e80 2024-09-10T18:57:12.659 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 >> v1:172.21.15.131:6790/0 conn(0x7f11401052f0 legacy=0x7f11401198a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:12.659 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 >> v1:172.21.15.175:6789/0 conn(0x7f1140102530 legacy=0x7f1140101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:12.659 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140121720 con 0x7f1140108e80 2024-09-10T18:57:12.659 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114011e750 con 0x7f1140108e80 2024-09-10T18:57:12.659 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f114011ec20 con 0x7f1140108e80 2024-09-10T18:57:12.659 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f1138002680 con 0x7f1140108e80 2024-09-10T18:57:12.660 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.650+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f1138005ca0 con 0x7f1140108e80 2024-09-10T18:57:12.660 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.651+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112c005010 con 0x7f1140108e80 2024-09-10T18:57:12.660 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.652+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f1138002230 con 0x7f1140108e80 2024-09-10T18:57:12.660 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.652+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 8 ==== osd_map(757..757 src has 1..757) v4 ==== 8198+0+0 (unknown 2954105327 0 0) 0x7f113808a8f0 con 0x7f1140108e80 2024-09-10T18:57:12.660 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.652+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=758}) v3 -- 0x7f1140101c60 con 0x7f1140108e80 2024-09-10T18:57:12.661 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.657+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f1138056e10 con 0x7f1140108e80 2024-09-10T18:57:12.814 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:12.812+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f112c0052c0 con 0x7f1140108e80 2024-09-10T18:57:13.225 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:13.224+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 10 ==== osd_map(758..758 src has 1..758) v4 ==== 620+0+0 (unknown 2335359748 0 0) 0x7f1138056b50 con 0x7f1140108e80 2024-09-10T18:57:13.225 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:13.224+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=759}) v3 -- 0x7f1140101e50 con 0x7f1140108e80 2024-09-10T18:57:13.227 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:13.226+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v758) v1 ==== 217+0+0 (unknown 1183226927 0 0) 0x7f11380567b0 con 0x7f1140108e80 2024-09-10T18:57:13.324 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:13.323+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f112c000be0 con 0x7f1140108e80 2024-09-10T18:57:13.576 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:13 smithi131 ceph-mon[40885]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:13.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:13 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:57:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:13 smithi131 ceph-mon[29369]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:13.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:13 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:57:13.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:13 smithi175 ceph-mon[33399]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:13.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:13 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:57:14.076 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:13] "GET /metrics HTTP/1.1" 200 228658 "" "Prometheus/2.43.0" 2024-09-10T18:57:14.230 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.228+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 12 ==== osd_map(759..759 src has 1..759) v4 ==== 620+0+0 (unknown 2233238660 0 0) 0x7f11380567b0 con 0x7f1140108e80 2024-09-10T18:57:14.230 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.228+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 --> v1:172.21.15.131:6789/0 -- mon_subscribe({osdmap=760}) v3 -- 0x7f1140101ad0 con 0x7f1140108e80 2024-09-10T18:57:14.233 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.232+0000 7f11277fe640 1 -- 172.21.15.131:0/1725276334 <== mon.0 v1:172.21.15.131:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v759) v1 ==== 217+0+0 (unknown 2547880417 0 0) 0x7f1138056990 con 0x7f1140108e80 2024-09-10T18:57:14.234 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_bytes = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:57:14.237 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.236+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 >> v1:172.21.15.131:6800/336444945 conn(0x7f112806fdb0 legacy=0x7f1128072260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:14.238 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.236+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 >> v1:172.21.15.131:6789/0 conn(0x7f1140108e80 legacy=0x7f114011cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:14.238 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.236+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 shutdown_connections 2024-09-10T18:57:14.238 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.236+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 >> 172.21.15.131:0/1725276334 conn(0x7f11400f8130 msgr2=0x7f1140109300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:14.238 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.236+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 shutdown_connections 2024-09-10T18:57:14.238 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.236+0000 7f11454ef640 1 -- 172.21.15.131:0/1725276334 wait complete. 2024-09-10T18:57:14.248 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool set-quota e7116ce6-52bb-4120-b144-3681a246036c max_objects 0 2024-09-10T18:57:14.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.328+0000 7f56da99d640 1 -- 172.21.15.131:0/3412441493 >> v1:172.21.15.131:6790/0 conn(0x7f56d41014a0 legacy=0x7f56d4101870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:14.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.328+0000 7f56da99d640 1 -- 172.21.15.131:0/3412441493 shutdown_connections 2024-09-10T18:57:14.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.328+0000 7f56da99d640 1 -- 172.21.15.131:0/3412441493 >> 172.21.15.131:0/3412441493 conn(0x7f56d40752f0 msgr2=0x7f56d40f94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:14.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.328+0000 7f56da99d640 1 -- 172.21.15.131:0/3412441493 shutdown_connections 2024-09-10T18:57:14.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.328+0000 7f56da99d640 1 -- 172.21.15.131:0/3412441493 wait complete. 2024-09-10T18:57:14.332 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56da99d640 1 Processor -- start 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56da99d640 1 -- start start 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56da99d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f56d411e160 con 0x7f56d4107d70 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56da99d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f56d411f320 con 0x7f56d4104210 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56da99d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f56d4120520 con 0x7f56d41014a0 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d3fff640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f56d41014a0 0x7f56d40646c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:51302/0 (socket says 172.21.15.131:51302) 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d3fff640 1 -- 172.21.15.131:0/2649805850 learned_addr learned my addr 172.21.15.131:0/2649805850 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d8f3a640 1 --1- 172.21.15.131:0/2649805850 >> v1:172.21.15.131:6789/0 conn(0x7f56d4107d70 0x7f56d411c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:55322/0 (socket says 172.21.15.131:55322) 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793438447 0 0) 0x7f56d4120520 con 0x7f56d41014a0 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56bc003200 con 0x7f56d41014a0 2024-09-10T18:57:14.333 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094315056 0 0) 0x7f56d411e160 con 0x7f56d4107d70 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d4120520 con 0x7f56d4107d70 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515072595 0 0) 0x7f56d411f320 con 0x7f56d4104210 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d411e160 con 0x7f56d4104210 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 911543450 0 0) 0x7f56bc003200 con 0x7f56d41014a0 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56d411f320 con 0x7f56d41014a0 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f56c4002c10 con 0x7f56d41014a0 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 515821813 0 0) 0x7f56d4120520 con 0x7f56d4107d70 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.329+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56bc003200 con 0x7f56d4107d70 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f56cc001e00 con 0x7f56d4107d70 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1352371758 0 0) 0x7f56d411e160 con 0x7f56d4104210 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f56d4120520 con 0x7f56d4104210 2024-09-10T18:57:14.334 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f56c80018c0 con 0x7f56d4104210 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3591916639 0 0) 0x7f56d411f320 con 0x7f56d41014a0 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 >> v1:172.21.15.175:6789/0 conn(0x7f56d4104210 legacy=0x7f56d4064dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 >> v1:172.21.15.131:6789/0 conn(0x7f56d4107d70 legacy=0x7f56d411c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4121720 con 0x7f56d41014a0 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d411e350 con 0x7f56d41014a0 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d411e7a0 con 0x7f56d41014a0 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f56c40022d0 con 0x7f56d41014a0 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.330+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f56c40034f0 con 0x7f56d41014a0 2024-09-10T18:57:14.335 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.331+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b4005010 con 0x7f56d41014a0 2024-09-10T18:57:14.336 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.331+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f56c4001e80 con 0x7f56d41014a0 2024-09-10T18:57:14.336 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.332+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(759..759 src has 1..759) v4 ==== 8198+0+0 (unknown 3388593330 0 0) 0x7f56c408a840 con 0x7f56d41014a0 2024-09-10T18:57:14.336 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.332+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=760}) v3 -- 0x7f56d411f320 con 0x7f56d41014a0 2024-09-10T18:57:14.337 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.336+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f56c4056d60 con 0x7f56d41014a0 2024-09-10T18:57:14.496 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:14.494+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f56b40052c0 con 0x7f56d41014a0 2024-09-10T18:57:14.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:57:14.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:14 smithi131 ceph-mon[29369]: osdmap e758: 8 total, 8 up, 8 in 2024-09-10T18:57:14.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:14 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:57:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:57:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:14 smithi131 ceph-mon[40885]: osdmap e758: 8 total, 8 up, 8 in 2024-09-10T18:57:14.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:14 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:57:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:57:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:14 smithi175 ceph-mon[33399]: osdmap e758: 8 total, 8 up, 8 in 2024-09-10T18:57:14.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:14 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T18:57:14.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:57:14 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:57:15.246 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:15.244+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 10 ==== osd_map(760..760 src has 1..760) v4 ==== 620+0+0 (unknown 3231997470 0 0) 0x7f56c4056aa0 con 0x7f56d41014a0 2024-09-10T18:57:15.246 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:15.245+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=761}) v3 -- 0x7f56bc003200 con 0x7f56d41014a0 2024-09-10T18:57:15.253 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:15.252+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v760) v1 ==== 221+0+0 (unknown 351141573 0 0) 0x7f56c4002440 con 0x7f56d41014a0 2024-09-10T18:57:15.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:15.351+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f56b4000c40 con 0x7f56d41014a0 2024-09-10T18:57:15.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:15] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:57:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-mon[33399]: osdmap e759: 8 total, 8 up, 8 in 2024-09-10T18:57:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2649805850' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-mon[33399]: from='client.57449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:15.427 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:15 smithi175 ceph-mon[33399]: pgmap v1247: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-10T18:57:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:57:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[29369]: osdmap e759: 8 total, 8 up, 8 in 2024-09-10T18:57:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2649805850' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[29369]: from='client.57449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:15.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[29369]: pgmap v1247: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-10T18:57:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1725276334' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T18:57:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[40885]: osdmap e759: 8 total, 8 up, 8 in 2024-09-10T18:57:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2649805850' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[40885]: from='client.57449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:15.578 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:15 smithi131 ceph-mon[40885]: pgmap v1247: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-09-10T18:57:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.238+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 12 ==== osd_map(761..761 src has 1..761) v4 ==== 620+0+0 (unknown 1019023030 0 0) 0x7f56c4057690 con 0x7f56d41014a0 2024-09-10T18:57:16.240 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.239+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=762}) v3 -- 0x7f56d411e160 con 0x7f56d41014a0 2024-09-10T18:57:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.243+0000 7f56d17fa640 1 -- 172.21.15.131:0/2649805850 <== mon.2 v1:172.21.15.131:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c v761) v1 ==== 221+0+0 (unknown 2495426340 0 0) 0x7f56c4056700 con 0x7f56d41014a0 2024-09-10T18:57:16.245 INFO:tasks.workunit.client.0.smithi131.stderr:set-quota max_objects = 0 for pool e7116ce6-52bb-4120-b144-3681a246036c 2024-09-10T18:57:16.248 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.247+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 >> v1:172.21.15.131:6800/336444945 conn(0x7f56bc06fd80 legacy=0x7f56bc072230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:16.248 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.247+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 >> v1:172.21.15.131:6790/0 conn(0x7f56d41014a0 legacy=0x7f56d40646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:16.248 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.247+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 shutdown_connections 2024-09-10T18:57:16.248 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.247+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 >> 172.21.15.131:0/2649805850 conn(0x7f56d40752f0 msgr2=0x7f56d40f9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:16.249 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.247+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 shutdown_connections 2024-09-10T18:57:16.249 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:16.247+0000 7f56da99d640 1 -- 172.21.15.131:0/2649805850 wait complete. 2024-09-10T18:57:16.258 INFO:tasks.workunit.client.0.smithi131.stderr:+ sleep 30 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[29369]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[29369]: osdmap e760: 8 total, 8 up, 8 in 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/2649805850' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[29369]: from='client.57449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[40885]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[40885]: osdmap e760: 8 total, 8 up, 8 in 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/2649805850' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:16.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:16 smithi131 ceph-mon[40885]: from='client.57449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:16.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:16 smithi175 ceph-mon[33399]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:57:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:16 smithi175 ceph-mon[33399]: osdmap e760: 8 total, 8 up, 8 in 2024-09-10T18:57:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:16 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/2649805850' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:16.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:16 smithi175 ceph-mon[33399]: from='client.57449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T18:57:17.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:17 smithi131 ceph-mon[29369]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:57:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:17 smithi131 ceph-mon[29369]: osdmap e761: 8 total, 8 up, 8 in 2024-09-10T18:57:17.577 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:17 smithi131 ceph-mon[29369]: pgmap v1250: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:57:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:17 smithi131 ceph-mon[40885]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:57:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:17 smithi131 ceph-mon[40885]: osdmap e761: 8 total, 8 up, 8 in 2024-09-10T18:57:17.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:17 smithi131 ceph-mon[40885]: pgmap v1250: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:57:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:17 smithi175 ceph-mon[33399]: from='client.57449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "field": "max_objects", "val": "0"}]': finished 2024-09-10T18:57:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:17 smithi175 ceph-mon[33399]: osdmap e761: 8 total, 8 up, 8 in 2024-09-10T18:57:17.677 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:17 smithi175 ceph-mon[33399]: pgmap v1250: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-09-10T18:57:18.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:18 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:18.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:18 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:18.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:18 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:19.576 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:19 smithi131 ceph-mon[29369]: pgmap v1251: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-10T18:57:19.577 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:19 smithi131 ceph-mon[40885]: pgmap v1251: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-10T18:57:19.676 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:19 smithi175 ceph-mon[33399]: pgmap v1251: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1 op/s 2024-09-10T18:57:21.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:21 smithi175 ceph-mon[33399]: pgmap v1252: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 813 B/s rd, 0 op/s 2024-09-10T18:57:22.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:21 smithi131 ceph-mon[29369]: pgmap v1252: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 813 B/s rd, 0 op/s 2024-09-10T18:57:22.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:21 smithi131 ceph-mon[40885]: pgmap v1252: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 813 B/s rd, 0 op/s 2024-09-10T18:57:23.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:23 smithi175 ceph-mon[33399]: pgmap v1253: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:57:24.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:23 smithi131 ceph-mon[29369]: pgmap v1253: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:57:24.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:23] "GET /metrics HTTP/1.1" 200 228625 "" "Prometheus/2.43.0" 2024-09-10T18:57:24.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:23 smithi131 ceph-mon[40885]: pgmap v1253: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:57:24.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:57:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:57:25.426 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:25] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:25.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:25 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:25.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:25 smithi175 ceph-mon[33399]: pgmap v1254: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:57:26.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:25 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:26.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:25 smithi131 ceph-mon[29369]: pgmap v1254: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:57:26.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:25 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:26.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:25 smithi131 ceph-mon[40885]: pgmap v1254: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:57:27.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:27 smithi175 ceph-mon[33399]: pgmap v1255: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-10T18:57:28.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:27 smithi131 ceph-mon[29369]: pgmap v1255: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-10T18:57:28.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:27 smithi131 ceph-mon[40885]: pgmap v1255: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 995 B/s rd, 0 op/s 2024-09-10T18:57:29.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:29 smithi175 ceph-mon[33399]: pgmap v1256: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:30.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:29 smithi131 ceph-mon[29369]: pgmap v1256: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:30.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:29 smithi131 ceph-mon[40885]: pgmap v1256: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:31.879 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:31 smithi131 ceph-mon[29369]: pgmap v1257: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:31.879 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:31 smithi131 ceph-mon[40885]: pgmap v1257: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:31.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:31 smithi175 ceph-mon[33399]: pgmap v1257: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:32.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:32 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:33.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:32 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:33.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:32 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:33.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:33 smithi175 ceph-mon[33399]: pgmap v1258: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:34.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:33 smithi131 ceph-mon[29369]: pgmap v1258: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:34.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:33] "GET /metrics HTTP/1.1" 200 228655 "" "Prometheus/2.43.0" 2024-09-10T18:57:34.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:33 smithi131 ceph-mon[40885]: pgmap v1258: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:34.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:57:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:57:35.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:35] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:35.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:35 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:35.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:35 smithi175 ceph-mon[33399]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:36.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:35 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:36.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:35 smithi131 ceph-mon[29369]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:36.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:35 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:36.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:35 smithi131 ceph-mon[40885]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:37.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:37 smithi175 ceph-mon[33399]: pgmap v1260: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:38.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:37 smithi131 ceph-mon[29369]: pgmap v1260: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:38.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:37 smithi131 ceph-mon[40885]: pgmap v1260: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:39.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:39 smithi175 ceph-mon[33399]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:40.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:39 smithi131 ceph-mon[29369]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:40.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:39 smithi131 ceph-mon[40885]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:41.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:41 smithi175 ceph-mon[33399]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:41.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:41 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:42.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:41 smithi131 ceph-mon[29369]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:42.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:41 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:41 smithi131 ceph-mon[40885]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:42.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:41 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:42.926 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:42 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:42 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:42.927 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:42 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:57:43.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:42 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:42 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:43.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:42 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:57:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:42 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:42 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:43.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:42 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' 2024-09-10T18:57:44.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:43 smithi131 ceph-mon[29369]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:44.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:43 smithi131 ceph-mon[40885]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:44.077 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:43] "GET /metrics HTTP/1.1" 200 228655 "" "Prometheus/2.43.0" 2024-09-10T18:57:44.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:43 smithi175 ceph-mon[33399]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:57:44.676 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:57:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug there is no tcmu-runner data available 2024-09-10T18:57:45.427 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x[35330]: ::ffff:172.21.15.175 - - [10/Sep/2024:18:57:45] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:46.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:45 smithi131 ceph-mon[29369]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:46.077 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:45 smithi131 ceph-mon[29369]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:45 smithi131 ceph-mon[40885]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:46.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:45 smithi131 ceph-mon[40885]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:46.176 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:45 smithi175 ceph-mon[33399]: from='client.34454 v1:172.21.15.175:0/2034261739' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:46.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:45 smithi175 ceph-mon[33399]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:46.260 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool delete e7116ce6-52bb-4120-b144-3681a246036c e7116ce6-52bb-4120-b144-3681a246036c --yes-i-really-really-mean-it 2024-09-10T18:57:46.349 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.345+0000 7f1449d9d640 1 -- 172.21.15.131:0/3598793954 >> v1:172.21.15.131:6789/0 conn(0x7f1444102530 legacy=0x7f1444102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.349 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.345+0000 7f1449d9d640 1 -- 172.21.15.131:0/3598793954 shutdown_connections 2024-09-10T18:57:46.349 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.345+0000 7f1449d9d640 1 -- 172.21.15.131:0/3598793954 >> 172.21.15.131:0/3598793954 conn(0x7f14440f8130 msgr2=0x7f14440fa540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:46.349 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.345+0000 7f1449d9d640 1 -- 172.21.15.131:0/3598793954 shutdown_connections 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.345+0000 7f1449d9d640 1 -- 172.21.15.131:0/3598793954 wait complete. 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1449d9d640 1 Processor -- start 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1449d9d640 1 -- start start 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1449d9d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f144411e140 con 0x7f1444102530 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1449d9d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f144411f300 con 0x7f14441052f0 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1449d9d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1444120500 con 0x7f1444108e80 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f14437fe640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f1444102530 0x7f1444106ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54290/0 (socket says 172.21.15.131:54290) 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f14437fe640 1 -- 172.21.15.131:0/1796306369 learned_addr learned my addr 172.21.15.131:0/1796306369 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1443fff640 1 --1- 172.21.15.131:0/1796306369 >> v1:172.21.15.131:6790/0 conn(0x7f1444108e80 0x7f144411c8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:39872/0 (socket says 172.21.15.131:39872) 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822184849 0 0) 0x7f144411e140 con 0x7f1444102530 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14300034e0 con 0x7f1444102530 2024-09-10T18:57:46.350 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822185154 0 0) 0x7f1444120500 con 0x7f1444108e80 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144411e140 con 0x7f1444108e80 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147385859 0 0) 0x7f144411f300 con 0x7f14441052f0 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.346+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1444120500 con 0x7f14441052f0 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 352982636 0 0) 0x7f14300034e0 con 0x7f1444102530 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f144411f300 con 0x7f1444102530 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f1434002ab0 con 0x7f1444102530 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1898863654 0 0) 0x7f144411e140 con 0x7f1444108e80 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14300034e0 con 0x7f1444108e80 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f143c001e00 con 0x7f1444108e80 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4052599301 0 0) 0x7f1444120500 con 0x7f14441052f0 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.175:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f144411e140 con 0x7f14441052f0 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f142c001840 con 0x7f14441052f0 2024-09-10T18:57:46.351 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1121320256 0 0) 0x7f14300034e0 con 0x7f1444108e80 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 >> v1:172.21.15.175:6789/0 conn(0x7f14441052f0 legacy=0x7f14441075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 >> v1:172.21.15.131:6789/0 conn(0x7f1444102530 legacy=0x7f1444106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444121700 con 0x7f1444108e80 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14441206f0 con 0x7f1444108e80 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1444120b40 con 0x7f1444108e80 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f143c001920 con 0x7f1444108e80 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.347+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f143c005bf0 con 0x7f1444108e80 2024-09-10T18:57:46.352 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.348+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1424005010 con 0x7f1444108e80 2024-09-10T18:57:46.353 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.348+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f143c005430 con 0x7f1444108e80 2024-09-10T18:57:46.353 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.349+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(761..761 src has 1..761) v4 ==== 8198+0+0 (unknown 2710894800 0 0) 0x7f143c08a770 con 0x7f1444108e80 2024-09-10T18:57:46.353 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.349+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=762}) v3 -- 0x7f14300034e0 con 0x7f1444108e80 2024-09-10T18:57:46.354 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.353+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f143c056c90 con 0x7f1444108e80 2024-09-10T18:57:46.510 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.509+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1424005840 con 0x7f1444108e80 2024-09-10T18:57:46.757 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.755+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 10 ==== osd_map(762..762 src has 1..762) v4 ==== 288+0+0 (unknown 617049916 0 0) 0x7f143c0569d0 con 0x7f1444108e80 2024-09-10T18:57:46.757 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.755+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=763}) v3 -- 0x7f144411f300 con 0x7f1444108e80 2024-09-10T18:57:46.761 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.759+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]=0 pool 'e7116ce6-52bb-4120-b144-3681a246036c' removed v762) v1 ==== 248+0+0 (unknown 377677933 0 0) 0x7f143c056630 con 0x7f1444108e80 2024-09-10T18:57:46.858 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.855+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1424005330 con 0x7f1444108e80 2024-09-10T18:57:46.858 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.856+0000 7f1440ff9640 1 -- 172.21.15.131:0/1796306369 <== mon.2 v1:172.21.15.131:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]=0 pool 'e7116ce6-52bb-4120-b144-3681a246036c' does not exist v762) v1 ==== 255+0+0 (unknown 1841237251 0 0) 0x7f143c056810 con 0x7f1444108e80 2024-09-10T18:57:46.858 INFO:tasks.workunit.client.0.smithi131.stderr:pool 'e7116ce6-52bb-4120-b144-3681a246036c' does not exist 2024-09-10T18:57:46.860 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.858+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 >> v1:172.21.15.131:6800/336444945 conn(0x7f143006fec0 legacy=0x7f1430072370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.861 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.858+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 >> v1:172.21.15.131:6790/0 conn(0x7f1444108e80 legacy=0x7f144411c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.861 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.858+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 shutdown_connections 2024-09-10T18:57:46.861 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.858+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 >> 172.21.15.131:0/1796306369 conn(0x7f14440f8130 msgr2=0x7f144410c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:46.861 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.858+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 shutdown_connections 2024-09-10T18:57:46.861 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.858+0000 7f1449d9d640 1 -- 172.21.15.131:0/1796306369 wait complete. 2024-09-10T18:57:46.869 INFO:tasks.workunit.client.0.smithi131.stderr:+ ceph osd pool delete 949e8d97-c4e4-4f93-ad36-26b7449a9224 949e8d97-c4e4-4f93-ad36-26b7449a9224 --yes-i-really-really-mean-it 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.943+0000 7f21ecd5d640 1 -- 172.21.15.131:0/1912381073 >> v1:172.21.15.131:6790/0 conn(0x7f21e81042a0 legacy=0x7f21e8106680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.943+0000 7f21ecd5d640 1 -- 172.21.15.131:0/1912381073 shutdown_connections 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.943+0000 7f21ecd5d640 1 -- 172.21.15.131:0/1912381073 >> 172.21.15.131:0/1912381073 conn(0x7f21e80f7030 msgr2=0x7f21e80f9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.943+0000 7f21ecd5d640 1 -- 172.21.15.131:0/1912381073 shutdown_connections 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.943+0000 7f21ecd5d640 1 -- 172.21.15.131:0/1912381073 wait complete. 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21ecd5d640 1 Processor -- start 2024-09-10T18:57:46.948 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21ecd5d640 1 -- start start 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21ecd5d640 1 -- --> v1:172.21.15.131:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21e811c030 con 0x7f21e81042a0 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21ecd5d640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21e811d1f0 con 0x7f21e8107e00 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21ecd5d640 1 -- --> v1:172.21.15.131:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f21e811e3f0 con 0x7f21e8101530 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21e659c640 1 --1- >> v1:172.21.15.131:6790/0 conn(0x7f21e8101530 0x7f21e8065c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6790/0 says I am v1:172.21.15.131:39890/0 (socket says 172.21.15.131:39890) 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21e659c640 1 -- 172.21.15.131:0/526936332 learned_addr learned my addr 172.21.15.131:0/526936332 (peer_addr_for_me v1:172.21.15.131:0/0) 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21e5d9b640 1 --1- >> v1:172.21.15.131:6789/0 conn(0x7f21e81042a0 0x7f21e8066380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.131:6789/0 says I am v1:172.21.15.131:54302/0 (socket says 172.21.15.131:54302) 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2927974724 0 0) 0x7f21e811e3f0 con 0x7f21e8101530 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d0003200 con 0x7f21e8101530 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.0 v1:172.21.15.131:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 507987127 0 0) 0x7f21e811c030 con 0x7f21e81042a0 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e811e3f0 con 0x7f21e81042a0 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2368322121 0 0) 0x7f21e811d1f0 con 0x7f21e8107e00 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.944+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e811c030 con 0x7f21e8107e00 2024-09-10T18:57:46.949 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1295228037 0 0) 0x7f21d0003200 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21e811d1f0 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f21d80019f0 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.0 v1:172.21.15.131:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2723135550 0 0) 0x7f21e811e3f0 con 0x7f21e81042a0 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f21d0003200 con 0x7f21e81042a0 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.0 v1:172.21.15.131:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f21dc002b50 con 0x7f21e81042a0 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1744567275 0 0) 0x7f21e811d1f0 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 >> v1:172.21.15.175:6789/0 conn(0x7f21e8107e00 legacy=0x7f21e811a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 >> v1:172.21.15.131:6789/0 conn(0x7f21e81042a0 legacy=0x7f21e8066380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e811f5f0 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e811c220 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e811c670 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 5 ==== config(23 keys) v1 ==== 994+0+0 (unknown 2809816869 0 0) 0x7f21d8002480 con 0x7f21e8101530 2024-09-10T18:57:46.950 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.945+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 2887432774 0 0) 0x7f21d8005ac0 con 0x7f21e8101530 2024-09-10T18:57:46.951 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.946+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c8005010 con 0x7f21e8101530 2024-09-10T18:57:46.951 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.946+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 7 ==== mgrmap(e 22) v1 ==== 90804+0+0 (unknown 2181218219 0 0) 0x7f21d8002030 con 0x7f21e8101530 2024-09-10T18:57:46.951 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.947+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 8 ==== osd_map(762..762 src has 1..762) v4 ==== 7810+0+0 (unknown 1273778748 0 0) 0x7f21d808a6a0 con 0x7f21e8101530 2024-09-10T18:57:46.951 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.947+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=763}) v3 -- 0x7f21e811d1f0 con 0x7f21e8101530 2024-09-10T18:57:46.953 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:46.951+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (unknown 1092875540 0 301564637) 0x7f21d8056d80 con 0x7f21e8101530 2024-09-10T18:57:47.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:46 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1796306369' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.076 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:46 smithi131 ceph-mon[29369]: from='client.57455 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:46 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1796306369' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.077 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:46 smithi131 ceph-mon[40885]: from='client.57455 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.110 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.108+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f21c8005840 con 0x7f21e8101530 2024-09-10T18:57:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:46 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1796306369' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:46 smithi175 ceph-mon[33399]: from='client.57455 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.767 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.765+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 10 ==== osd_map(763..763 src has 1..763) v4 ==== 288+0+0 (unknown 4117894868 0 0) 0x7f21d8056ac0 con 0x7f21e8101530 2024-09-10T18:57:47.768 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.765+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_subscribe({osdmap=764}) v3 -- 0x7f21e811e3f0 con 0x7f21e8101530 2024-09-10T18:57:47.770 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.768+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]=0 pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' removed v763) v1 ==== 248+0+0 (unknown 2951163258 0 0) 0x7f21d8056720 con 0x7f21e8101530 2024-09-10T18:57:47.867 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.865+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 --> v1:172.21.15.131:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f21c8000c10 con 0x7f21e8101530 2024-09-10T18:57:47.868 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.866+0000 7f21d6ffd640 1 -- 172.21.15.131:0/526936332 <== mon.2 v1:172.21.15.131:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]=0 pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' does not exist v763) v1 ==== 255+0+0 (unknown 3266835960 0 0) 0x7f21d8056900 con 0x7f21e8101530 2024-09-10T18:57:47.868 INFO:tasks.workunit.client.0.smithi131.stderr:pool '949e8d97-c4e4-4f93-ad36-26b7449a9224' does not exist 2024-09-10T18:57:47.870 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.868+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 >> v1:172.21.15.131:6800/336444945 conn(0x7f21d006fcb0 legacy=0x7f21d0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:47.870 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.868+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 >> v1:172.21.15.131:6790/0 conn(0x7f21e8101530 legacy=0x7f21e8065c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:57:47.870 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.868+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 shutdown_connections 2024-09-10T18:57:47.870 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.868+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 >> 172.21.15.131:0/526936332 conn(0x7f21e80f7030 msgr2=0x7f21e80fbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:57:47.870 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.868+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 shutdown_connections 2024-09-10T18:57:47.871 INFO:tasks.workunit.client.0.smithi131.stderr:2024-09-10T18:57:47.868+0000 7f21ecd5d640 1 -- 172.21.15.131:0/526936332 wait complete. 2024-09-10T18:57:47.879 INFO:tasks.workunit.client.0.smithi131.stderr:+ echo OK 2024-09-10T18:57:47.880 INFO:tasks.workunit.client.0.smithi131.stdout:OK 2024-09-10T18:57:47.880 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T18:57:47.881 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T18:57:47.947 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-10T18:57:47.947 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T18:57:48.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:48.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: from='client.57455 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:57:48.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: osdmap e762: 8 total, 8 up, 8 in 2024-09-10T18:57:48.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/1796306369' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: from='client.57455 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.009 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/526936332' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: from='client.57461 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[29369]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: from='client.57455 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: osdmap e762: 8 total, 8 up, 8 in 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/1796306369' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: from='client.57455 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/526936332' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: from='client.57461 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.010 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:47 smithi131 ceph-mon[40885]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: from='client.57455 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: osdmap e762: 8 total, 8 up, 8 in 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/1796306369' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: from='client.57455 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e7116ce6-52bb-4120-b144-3681a246036c", "pool2": "e7116ce6-52bb-4120-b144-3681a246036c", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/526936332' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: from='client.57461 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:47 smithi175 ceph-mon[33399]: from='mgr.34496 v1:172.21.15.131:0/2465796223' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:48.266 DEBUG:teuthology.parallel:result is None 2024-09-10T18:57:48.267 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:57:48.331 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:57:48.332 DEBUG:teuthology.orchestra.run.smithi131:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:57:48.393 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T18:57:48.394 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T18:57:48.403 INFO:tasks.cephadm:Teardown begin 2024-09-10T18:57:48.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:48.460 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:57:48.494 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T18:57:48.494 DEBUG:teuthology.orchestra.run.smithi131:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:57:48.515 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:57:48.552 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T18:57:48.552 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T18:57:48.552 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:57:48.876 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 systemd[1]: Stopping Ceph mon.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[29369]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[29369]: from='client.57461 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[29369]: osdmap e763: 8 total, 8 up, 8 in 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[29369]: from='client.? v1:172.21.15.131:0/526936332' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[29369]: from='client.57461 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[40885]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[40885]: from='client.57461 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[40885]: osdmap e763: 8 total, 8 up, 8 in 2024-09-10T18:57:48.877 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[40885]: from='client.? v1:172.21.15.131:0/526936332' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.878 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-mon[40885]: from='client.57461 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:49.141 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:57:48] ENGINE Bus STOPPING 2024-09-10T18:57:49.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a[29346]: 2024-09-10T18:57:48.875+0000 7fac48175640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:57:49.142 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:48 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a[29346]: 2024-09-10T18:57:48.875+0000 7fac48175640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T18:57:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:48 smithi175 ceph-mon[33399]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:57:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:48 smithi175 ceph-mon[33399]: from='client.57461 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:57:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:48 smithi175 ceph-mon[33399]: osdmap e763: 8 total, 8 up, 8 in 2024-09-10T18:57:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:48 smithi175 ceph-mon[33399]: from='client.? v1:172.21.15.131:0/526936332' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:49.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:48 smithi175 ceph-mon[33399]: from='client.57461 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "pool2": "949e8d97-c4e4-4f93-ad36-26b7449a9224", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:57:49.420 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:49 smithi131 podman[150303]: 2024-09-10 18:57:49.141066494 +0000 UTC m=+0.392023279 container died e35573a25943145ac0777f4f0facf5fe0e3e19af402f02b277b94ac5b9fd912d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T18:57:49.420 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:49 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:57:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:57:49.420 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:49 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:57:49] ENGINE Bus STOPPED 2024-09-10T18:57:49.420 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:49 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:57:49] ENGINE Bus STARTING 2024-09-10T18:57:49.826 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:49 smithi131 podman[150303]: 2024-09-10 18:57:49.624684935 +0000 UTC m=+0.875641768 container cleanup e35573a25943145ac0777f4f0facf5fe0e3e19af402f02b277b94ac5b9fd912d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:57:49.827 INFO:journalctl@ceph.mon.a.smithi131.stdout:Sep 10 18:57:49 smithi131 bash[150303]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:57:49.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:49 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:57:49] ENGINE Serving on http://:::9283 2024-09-10T18:57:49.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:49 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: [10/Sep/2024:18:57:49] ENGINE Bus STARTED 2024-09-10T18:57:50.055 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T18:57:50.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:50.092 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T18:57:50.093 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T18:57:50.093 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.c 2024-09-10T18:57:50.423 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:50 smithi131 systemd[1]: Stopping Ceph mon.c for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:57:50.826 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c[40862]: 2024-09-10T18:57:50.421+0000 7fad4ced1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:57:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c[40862]: 2024-09-10T18:57:50.421+0000 7fad4ced1640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:57:50.827 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:50 smithi131 podman[150449]: 2024-09-10 18:57:50.576332899 +0000 UTC m=+0.259824249 container died e89026a64335aed583dcd9035af67e5db79a2f30050eba7d5ecf2ee40dbade3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T18:57:51.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:50 smithi131 podman[150449]: 2024-09-10 18:57:50.945949842 +0000 UTC m=+0.629441190 container cleanup e89026a64335aed583dcd9035af67e5db79a2f30050eba7d5ecf2ee40dbade3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:57:51.214 INFO:journalctl@ceph.mon.c.smithi131.stdout:Sep 10 18:57:50 smithi131 bash[150449]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-c 2024-09-10T18:57:51.376 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T18:57:51.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:51.411 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T18:57:51.412 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T18:57:51.412 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.b 2024-09-10T18:57:51.778 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:51 smithi175 systemd[1]: Stopping Ceph mon.b for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:57:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b[33355]: 2024-09-10T18:57:51.776+0000 7f3aa9027640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:57:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b[33355]: 2024-09-10T18:57:51.776+0000 7f3aa9027640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:57:52.177 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:51 smithi175 podman[72921]: 2024-09-10 18:57:51.91050958 +0000 UTC m=+0.298614369 container died 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-10T18:57:52.636 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:52 smithi175 podman[72921]: 2024-09-10 18:57:52.426456908 +0000 UTC m=+0.814561694 container cleanup 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T18:57:52.637 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:52 smithi175 bash[72921]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b 2024-09-10T18:57:52.865 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T18:57:52.887 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:52 smithi175 podman[72938]: 2024-09-10 18:57:52.636096452 +0000 UTC m=+0.720834033 container remove 4ae2ee802e7b1dc10eaebbf3d070e428918afc762b11a3096003a3ff451ebf76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:57:52.887 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:52 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-10T18:57:52.888 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:52 smithi175 systemd[1]: Stopped Ceph mon.b for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:57:52.888 INFO:journalctl@ceph.mon.b.smithi175.stdout:Sep 10 18:57:52 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mon.b.service: Consumed 24.587s CPU time. 2024-09-10T18:57:52.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:52.916 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T18:57:52.916 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T18:57:52.916 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:57:53.228 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:52 smithi131 systemd[1]: Stopping Ceph mgr.y for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:57:53.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:57:53.226+0000 7f47267d5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:57:53.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y[29621]: 2024-09-10T18:57:53.226+0000 7f47267d5640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T18:57:53.536 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:53 smithi131 podman[150582]: 2024-09-10 18:57:53.258826335 +0000 UTC m=+0.136874438 container died 112b87f9c85b1a1507dc6d6c0aedff8b91d3492e18507602f3c945334dbcad36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:57:53.826 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:53 smithi131 podman[150582]: 2024-09-10 18:57:53.634122272 +0000 UTC m=+0.512170364 container cleanup 112b87f9c85b1a1507dc6d6c0aedff8b91d3492e18507602f3c945334dbcad36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T18:57:53.827 INFO:journalctl@ceph.mgr.y.smithi131.stdout:Sep 10 18:57:53 smithi131 bash[150582]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-y 2024-09-10T18:57:54.064 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T18:57:54.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:54.099 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T18:57:54.100 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T18:57:54.100 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.x 2024-09-10T18:57:54.413 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:54 smithi175 systemd[1]: Stopping Ceph mgr.x for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:54 smithi175 podman[73054]: 2024-09-10 18:57:54.412643615 +0000 UTC m=+0.131726967 container died 48d6e92c930db6e25bd4c3373e3a881e5bf515d0ec089631ebafa3d57b20445a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T18:57:54.926 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:54 smithi175 podman[73054]: 2024-09-10 18:57:54.813693652 +0000 UTC m=+0.532776994 container cleanup 48d6e92c930db6e25bd4c3373e3a881e5bf515d0ec089631ebafa3d57b20445a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T18:57:54.927 INFO:journalctl@ceph.mgr.x.smithi175.stdout:Sep 10 18:57:54 smithi175 bash[73054]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-mgr-x 2024-09-10T18:57:55.173 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T18:57:55.210 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:57:55.210 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T18:57:55.210 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T18:57:55.210 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:57:55.516 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:57:55 smithi131 systemd[1]: Stopping Ceph osd.0 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:57:55.826 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:57:55 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0[45274]: 2024-09-10T18:57:55.514+0000 7f16b69a0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:57:55.826 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:57:55 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0[45274]: 2024-09-10T18:57:55.514+0000 7f16b69a0640 -1 osd.0 763 *** Got signal Terminated *** 2024-09-10T18:57:55.827 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:57:55 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0[45274]: 2024-09-10T18:57:55.514+0000 7f16b69a0640 -1 osd.0 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:00.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:00.148Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:00.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:00.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:01.147 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:00 smithi131 podman[150713]: 2024-09-10 18:58:00.850059648 +0000 UTC m=+5.448392149 container died 88674cb7fb446cf8450d05fd2b00ea68b832af123d740dcae3cdf7dc580ee844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:58:01.577 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 podman[150713]: 2024-09-10 18:58:01.146673095 +0000 UTC m=+5.745005586 container cleanup 88674cb7fb446cf8450d05fd2b00ea68b832af123d740dcae3cdf7dc580ee844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T18:58:01.577 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 bash[150713]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:58:01.577 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 podman[150735]: 2024-09-10 18:58:01.390008673 +0000 UTC m=+0.533922512 container remove 88674cb7fb446cf8450d05fd2b00ea68b832af123d740dcae3cdf7dc580ee844 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:58:01.952 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 podman[150828]: 2024-09-10 18:58:01.691075242 +0000 UTC m=+0.092151400 container create 7ae0fcff244e38de1e6eac74999b2302e2b98352b84b5e2cdb6f47784e142429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:58:01.952 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 podman[150828]: 2024-09-10 18:58:01.614664655 +0000 UTC m=+0.015740771 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:01.952 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 podman[150828]: 2024-09-10 18:58:01.951950318 +0000 UTC m=+0.353026450 container init 7ae0fcff244e38de1e6eac74999b2302e2b98352b84b5e2cdb6f47784e142429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:58:02.327 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:01 smithi131 podman[150828]: 2024-09-10 18:58:01.95704669 +0000 UTC m=+0.358122813 container start 7ae0fcff244e38de1e6eac74999b2302e2b98352b84b5e2cdb6f47784e142429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:58:02.327 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:02 smithi131 podman[150828]: 2024-09-10 18:58:02.052283415 +0000 UTC m=+0.453359534 container attach 7ae0fcff244e38de1e6eac74999b2302e2b98352b84b5e2cdb6f47784e142429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True) 2024-09-10T18:58:02.327 INFO:journalctl@ceph.osd.0.smithi131.stdout:Sep 10 18:58:02 smithi131 podman[150828]: 2024-09-10 18:58:02.136906077 +0000 UTC m=+0.537982201 container died 7ae0fcff244e38de1e6eac74999b2302e2b98352b84b5e2cdb6f47784e142429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-0-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T18:58:02.785 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T18:58:02.825 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:02.825 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T18:58:02.826 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T18:58:02.826 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.1 2024-09-10T18:58:03.076 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:02 smithi131 systemd[1]: Stopping Ceph osd.1 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:03.576 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1[51510]: 2024-09-10T18:58:03.177+0000 7fd37cc5b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:03.577 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1[51510]: 2024-09-10T18:58:03.177+0000 7fd37cc5b640 -1 osd.1 763 *** Got signal Terminated *** 2024-09-10T18:58:03.577 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1[51510]: 2024-09-10T18:58:03.177+0000 7fd37cc5b640 -1 osd.1 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:08.827 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:08 smithi131 podman[150957]: 2024-09-10 18:58:08.561732538 +0000 UTC m=+5.518545124 container died 6dc7b7d2f322ce118a9d073fbfffb305f68dcbd314615070bb95a24e5d147dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:58:09.269 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:08 smithi131 podman[150957]: 2024-09-10 18:58:08.959270522 +0000 UTC m=+5.916083108 container cleanup 6dc7b7d2f322ce118a9d073fbfffb305f68dcbd314615070bb95a24e5d147dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:58:09.270 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:08 smithi131 bash[150957]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1 2024-09-10T18:58:09.576 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:09 smithi131 podman[150994]: 2024-09-10 18:58:09.269053007 +0000 UTC m=+0.705211918 container remove 6dc7b7d2f322ce118a9d073fbfffb305f68dcbd314615070bb95a24e5d147dd5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T18:58:09.948 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:09 smithi131 podman[151084]: 2024-09-10 18:58:09.521111933 +0000 UTC m=+0.015702912 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:09.948 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:09 smithi131 podman[151084]: 2024-09-10 18:58:09.620031777 +0000 UTC m=+0.114622793 container create 95edb643d528feeef256650d1039a41afba27997cc76f72ab5311be9a82b9501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:58:09.948 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:09 smithi131 podman[151084]: 2024-09-10 18:58:09.839540594 +0000 UTC m=+0.334131572 container init 95edb643d528feeef256650d1039a41afba27997cc76f72ab5311be9a82b9501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T18:58:09.949 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:09 smithi131 podman[151084]: 2024-09-10 18:58:09.845058967 +0000 UTC m=+0.339649937 container start 95edb643d528feeef256650d1039a41afba27997cc76f72ab5311be9a82b9501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T18:58:09.949 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:09 smithi131 podman[151084]: 2024-09-10 18:58:09.947700278 +0000 UTC m=+0.442291249 container attach 95edb643d528feeef256650d1039a41afba27997cc76f72ab5311be9a82b9501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:58:10.326 INFO:journalctl@ceph.osd.1.smithi131.stdout:Sep 10 18:58:10 smithi131 podman[151084]: 2024-09-10 18:58:10.030566825 +0000 UTC m=+0.525157796 container died 95edb643d528feeef256650d1039a41afba27997cc76f72ab5311be9a82b9501 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T18:58:10.327 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:10 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:10.149Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:10.327 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:10 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:10.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:10.327 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:10 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:10.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:10.647 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T18:58:10.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:10.685 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T18:58:10.685 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T18:58:10.686 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.2 2024-09-10T18:58:11.015 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:10 smithi131 systemd[1]: Stopping Ceph osd.2 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:11.326 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:11 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:58:11.014+0000 7f6e8f90a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:11.326 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:11 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:58:11.014+0000 7f6e8f90a640 -1 osd.2 763 *** Got signal Terminated *** 2024-09-10T18:58:11.326 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:11 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2[57904]: 2024-09-10T18:58:11.014+0000 7f6e8f90a640 -1 osd.2 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:13.826 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:13.571Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:13.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:13 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:13.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:16.814 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:16 smithi131 podman[151213]: 2024-09-10 18:58:16.506142027 +0000 UTC m=+5.597408287 container died a31efdc09d44764b5d22bed3ff6080f04cc09a9ce6f35c2cf383a66f49e25c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:58:17.076 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:16 smithi131 podman[151213]: 2024-09-10 18:58:16.912777218 +0000 UTC m=+6.004043477 container cleanup a31efdc09d44764b5d22bed3ff6080f04cc09a9ce6f35c2cf383a66f49e25c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T18:58:17.077 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:16 smithi131 bash[151213]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2 2024-09-10T18:58:17.474 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151233]: 2024-09-10 18:58:17.147554169 +0000 UTC m=+0.642987095 container remove a31efdc09d44764b5d22bed3ff6080f04cc09a9ce6f35c2cf383a66f49e25c62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T18:58:17.725 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151329]: 2024-09-10 18:58:17.375881479 +0000 UTC m=+0.015664426 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:17.725 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151329]: 2024-09-10 18:58:17.498301013 +0000 UTC m=+0.138083956 container create a17235e2dd4d4d9f1c41cb2f5b7f8d72df6cc33aebe9ba7f73c9ceefc29c28c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T18:58:17.725 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151329]: 2024-09-10 18:58:17.723908582 +0000 UTC m=+0.363691533 container init a17235e2dd4d4d9f1c41cb2f5b7f8d72df6cc33aebe9ba7f73c9ceefc29c28c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:58:18.077 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151329]: 2024-09-10 18:58:17.729362051 +0000 UTC m=+0.369144996 container start a17235e2dd4d4d9f1c41cb2f5b7f8d72df6cc33aebe9ba7f73c9ceefc29c28c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T18:58:18.077 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151329]: 2024-09-10 18:58:17.808907571 +0000 UTC m=+0.448690515 container attach a17235e2dd4d4d9f1c41cb2f5b7f8d72df6cc33aebe9ba7f73c9ceefc29c28c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T18:58:18.077 INFO:journalctl@ceph.osd.2.smithi131.stdout:Sep 10 18:58:17 smithi131 podman[151329]: 2024-09-10 18:58:17.912564077 +0000 UTC m=+0.552347021 container died a17235e2dd4d4d9f1c41cb2f5b7f8d72df6cc33aebe9ba7f73c9ceefc29c28c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-2-deactivate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T18:58:18.510 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T18:58:18.548 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:18.549 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T18:58:18.549 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T18:58:18.549 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.3 2024-09-10T18:58:18.827 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:18 smithi131 systemd[1]: Stopping Ceph osd.3 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:19.326 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:58:18.884+0000 7fbb394c0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:19.327 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:58:18.884+0000 7fbb394c0640 -1 osd.3 763 *** Got signal Terminated *** 2024-09-10T18:58:19.327 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:18 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:58:18.884+0000 7fbb394c0640 -1 osd.3 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:20.576 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:20 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:20.150Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:20.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:20 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:20.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:20.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:20 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:20.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:21.677 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:21 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:21.242+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:21.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:21 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:21.287+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:22.676 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:22 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:22.288+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:22.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:22 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:22.487+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:22.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:22 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:22.284+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:22.826 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:22 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:58:22.419+0000 7fbb352d7640 -1 osd.3 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.796579+0000 front 2024-09-10T18:57:59.796689+0000 (oldest deadline 2024-09-10T18:58:22.096775+0000) 2024-09-10T18:58:23.519 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:23.259+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:23.520 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:23.274+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:23.826 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3[64052]: 2024-09-10T18:58:23.411+0000 7fbb352d7640 -1 osd.3 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.796579+0000 front 2024-09-10T18:57:59.796689+0000 (oldest deadline 2024-09-10T18:58:22.096775+0000) 2024-09-10T18:58:23.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:23.571Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:23.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:23.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:23.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:23 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:23.572Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:23.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:23.642+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:23.927 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:23 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:23.518+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:24.500 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:24 smithi131 podman[151456]: 2024-09-10 18:58:24.243729738 +0000 UTC m=+5.477068919 container died 8782ff4aa24f5317d080a5ea30c31cbe8e8d3562fb306e105a0bbed0e65f25ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:58:24.629 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:24.264+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:24.630 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:24.474+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:24.630 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:24.255+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:24.775 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:24 smithi131 podman[151456]: 2024-09-10 18:58:24.598886535 +0000 UTC m=+5.832225715 container cleanup 8782ff4aa24f5317d080a5ea30c31cbe8e8d3562fb306e105a0bbed0e65f25ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T18:58:24.776 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:24 smithi131 bash[151456]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3 2024-09-10T18:58:24.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:24 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:24.628+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:25.076 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:24 smithi131 podman[151480]: 2024-09-10 18:58:24.774938727 +0000 UTC m=+0.523601675 container remove 8782ff4aa24f5317d080a5ea30c31cbe8e8d3562fb306e105a0bbed0e65f25ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T18:58:25.520 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:25 smithi131 podman[151569]: 2024-09-10 18:58:25.20079597 +0000 UTC m=+0.109568006 container create cb9ef890e52a19ea9f40130596813ab3a6e37c0972119e4f8aed7d3428a7282e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T18:58:25.521 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:25 smithi131 podman[151569]: 2024-09-10 18:58:25.106626068 +0000 UTC m=+0.015398113 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:25.521 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:25 smithi131 podman[151569]: 2024-09-10 18:58:25.419860433 +0000 UTC m=+0.328632467 container init cb9ef890e52a19ea9f40130596813ab3a6e37c0972119e4f8aed7d3428a7282e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:58:25.521 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:25 smithi131 podman[151569]: 2024-09-10 18:58:25.424804447 +0000 UTC m=+0.333576491 container start cb9ef890e52a19ea9f40130596813ab3a6e37c0972119e4f8aed7d3428a7282e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T18:58:25.521 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:25 smithi131 podman[151569]: 2024-09-10 18:58:25.519958671 +0000 UTC m=+0.428730723 container attach cb9ef890e52a19ea9f40130596813ab3a6e37c0972119e4f8aed7d3428a7282e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T18:58:25.671 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:25.449+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:25.672 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:25.240+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:25.672 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:25.255+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:25.827 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:25 smithi131 podman[151569]: 2024-09-10 18:58:25.599355175 +0000 UTC m=+0.508127227 container died cb9ef890e52a19ea9f40130596813ab3a6e37c0972119e4f8aed7d3428a7282e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:58:25.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:25 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:25.670+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:26.183 INFO:journalctl@ceph.osd.3.smithi131.stdout:Sep 10 18:58:26 smithi131 podman[151612]: 2024-09-10 18:58:26.164996798 +0000 UTC m=+0.560426636 container remove cb9ef890e52a19ea9f40130596813ab3a6e37c0972119e4f8aed7d3428a7282e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:58:26.188 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T18:58:26.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:26.226 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T18:58:26.226 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T18:58:26.226 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.4 2024-09-10T18:58:26.534 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:26 smithi175 systemd[1]: Stopping Ceph osd.4 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:26.534 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:26.284+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:26.534 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:26.532+0000 7fb398b4b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:26.534 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:26.431+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:26.535 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:26.213+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:26.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:26.683+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:26.927 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:26.532+0000 7fb398b4b640 -1 osd.4 763 *** Got signal Terminated *** 2024-09-10T18:58:26.927 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:26 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:26.532+0000 7fb398b4b640 -1 osd.4 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:27.676 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:27 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:27.314+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:27.676 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:27 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:27.219+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:27.676 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:27 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:27.448+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:27.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:27 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:27.674+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:28.646 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:28 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:28.451+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:28.647 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:28 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:28.330+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:28.647 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:28 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:28.249+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:28.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:28 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:28.645+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:29.611 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:29 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:29.342+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:29.611 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:29 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:29.273+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:29.612 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:29 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:29.467+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:29.612 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:29 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:29.467+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:29.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:29 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:29.609+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:29.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:29 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:29.609+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:30.576 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:30 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:30.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:30.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:30 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:30.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:30.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:30 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:30.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:30.656 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:30 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:30.382+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:30.657 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:30 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:30.424+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:30.657 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:30 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:30.424+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:30.657 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:30 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:30.284+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:30.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:30 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:30.655+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:30.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:30 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:30.655+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:31.609 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4[38678]: 2024-09-10T18:58:31.351+0000 7fb394161640 -1 osd.4 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:55.896878+0000 front 2024-09-10T18:57:55.896900+0000 (oldest deadline 2024-09-10T18:58:21.196895+0000) 2024-09-10T18:58:31.609 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:31.380+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:31.609 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:31.380+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:31.610 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:31.267+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:31.610 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:31.267+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:31.904 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:31.608+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:31.904 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:31 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:31.608+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:32.177 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:31 smithi175 podman[73204]: 2024-09-10 18:58:31.903629978 +0000 UTC m=+5.482847949 container died 9c588ac1b6dc6fd8149df5d53aa634833a0f907d6ac7abad60b6d2487ccccf92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T18:58:32.541 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:32 smithi175 podman[73204]: 2024-09-10 18:58:32.288672128 +0000 UTC m=+5.867890091 container cleanup 9c588ac1b6dc6fd8149df5d53aa634833a0f907d6ac7abad60b6d2487ccccf92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T18:58:32.541 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:32 smithi175 bash[73204]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4 2024-09-10T18:58:32.541 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:32 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:32.350+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:32.541 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:32 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:32.350+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:32.541 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:32 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:32.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:32.541 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:32 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:32.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:32.841 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:32 smithi175 podman[73224]: 2024-09-10 18:58:32.539870816 +0000 UTC m=+0.638527549 container remove 9c588ac1b6dc6fd8149df5d53aa634833a0f907d6ac7abad60b6d2487ccccf92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T18:58:32.841 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:32 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:32.565+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:32.842 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:32 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:32.565+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:33.135 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:32 smithi175 podman[73319]: 2024-09-10 18:58:32.840783263 +0000 UTC m=+0.099033845 container create a9c407461b3d3d775079d611e32dea45264e0f9dd2f64a90bcd29994d6f31e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T18:58:33.135 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:32 smithi175 podman[73319]: 2024-09-10 18:58:32.757903021 +0000 UTC m=+0.016153614 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:33.136 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:33 smithi175 podman[73319]: 2024-09-10 18:58:33.076697822 +0000 UTC m=+0.334948408 container init a9c407461b3d3d775079d611e32dea45264e0f9dd2f64a90bcd29994d6f31e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:58:33.136 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:33 smithi175 podman[73319]: 2024-09-10 18:58:33.081386744 +0000 UTC m=+0.339637327 container start a9c407461b3d3d775079d611e32dea45264e0f9dd2f64a90bcd29994d6f31e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T18:58:33.136 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:33 smithi175 podman[73319]: 2024-09-10 18:58:33.134868576 +0000 UTC m=+0.393119162 container attach a9c407461b3d3d775079d611e32dea45264e0f9dd2f64a90bcd29994d6f31e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:58:33.426 INFO:journalctl@ceph.osd.4.smithi175.stdout:Sep 10 18:58:33 smithi175 podman[73319]: 2024-09-10 18:58:33.265464682 +0000 UTC m=+0.523715275 container died a9c407461b3d3d775079d611e32dea45264e0f9dd2f64a90bcd29994d6f31e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T18:58:33.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:33.323+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:33.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:33.323+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:33.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:33.237+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:33.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:33.237+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:33.822 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:33.594+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:33.822 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:33 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:33.594+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:33.826 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:33.572Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:33.826 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:33.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:33.826 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:33 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:33.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:33.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T18:58:33.881 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:33.881 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T18:58:33.881 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T18:58:33.881 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.5 2024-09-10T18:58:34.177 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:33 smithi175 systemd[1]: Stopping Ceph osd.5 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:34.592 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:34.321+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:34.592 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:34.321+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:34.592 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:34.203+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:34.592 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:34.203+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:34.592 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:34.218+0000 7f3861ae1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:34.592 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:34.218+0000 7f3861ae1640 -1 osd.5 763 *** Got signal Terminated *** 2024-09-10T18:58:34.593 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:34.218+0000 7f3861ae1640 -1 osd.5 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:34.593 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:34.590+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:34.593 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:34 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:34.590+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:35.614 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:35.345+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:35.615 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:35.345+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:35.615 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:35.231+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:35.615 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:35.231+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:35.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:35.613+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:35.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:35 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:35.613+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:36.569 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:36 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:36.336+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:36.569 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:36 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:36.336+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:36.569 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:36 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:36.209+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:36.569 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:36 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:36.209+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:36.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:36 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:36.568+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:36.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:36 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:36.568+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:37.532 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:37.354+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:37.532 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:37.354+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:37.532 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:37.205+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:37.532 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:37.205+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:37.926 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:37.531+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:37.927 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:37 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:37.531+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:38.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:38.396+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:38.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:38.396+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:38.678 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:38.488+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.554636+0000 front 2024-09-10T18:57:59.554709+0000 (oldest deadline 2024-09-10T18:58:23.054582+0000) 2024-09-10T18:58:38.678 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:38.488+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:03.055034+0000 front 2024-09-10T18:58:03.055113+0000 (oldest deadline 2024-09-10T18:58:28.954977+0000) 2024-09-10T18:58:38.678 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5[44994]: 2024-09-10T18:58:38.488+0000 7f385d0f7640 -1 osd.5 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:14.256156+0000 front 2024-09-10T18:58:14.256126+0000 (oldest deadline 2024-09-10T18:58:37.756071+0000) 2024-09-10T18:58:38.678 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:38.246+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:38.679 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:38.246+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:38.679 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:38 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:38.246+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:39.626 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:39 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:39.348+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:39.626 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:39 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:39.348+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:39.626 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:39 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:39.238+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:39.626 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:39 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:39.238+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:39.626 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:39 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:39.238+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:39.892 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:39 smithi175 podman[73450]: 2024-09-10 18:58:39.625308269 +0000 UTC m=+5.523577997 container died 0e6805b8d61868d1ac25c67dd7a827f8541bc98a1e715fc9c420ffb3954bcfe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:58:40.176 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:39 smithi175 podman[73450]: 2024-09-10 18:58:39.973522696 +0000 UTC m=+5.871792423 container cleanup 0e6805b8d61868d1ac25c67dd7a827f8541bc98a1e715fc9c420ffb3954bcfe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T18:58:40.177 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:39 smithi175 bash[73450]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5 2024-09-10T18:58:40.544 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73471]: 2024-09-10 18:58:40.235243973 +0000 UTC m=+0.612586612 container remove 0e6805b8d61868d1ac25c67dd7a827f8541bc98a1e715fc9c420ffb3954bcfe6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T18:58:40.545 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:40 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:40.327+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:40.545 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:40 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:40.327+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:40.545 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:40 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:40.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:40.545 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:40 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:40.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:40.545 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:40 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:40.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:40.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:40 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:40.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:40.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:40 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:40.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:40.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:40 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:40.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:40.839 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73564]: 2024-09-10 18:58:40.544289521 +0000 UTC m=+0.112798778 container create a960f4afcc807ed0eae82e3a5b95ae8017f4a3a91be2d4d728fb8b9f1942fc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T18:58:40.839 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73564]: 2024-09-10 18:58:40.448353957 +0000 UTC m=+0.016863223 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:40.839 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73564]: 2024-09-10 18:58:40.754948287 +0000 UTC m=+0.323457549 container init a960f4afcc807ed0eae82e3a5b95ae8017f4a3a91be2d4d728fb8b9f1942fc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:58:40.839 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73564]: 2024-09-10 18:58:40.759779013 +0000 UTC m=+0.328288271 container start a960f4afcc807ed0eae82e3a5b95ae8017f4a3a91be2d4d728fb8b9f1942fc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:58:41.175 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73564]: 2024-09-10 18:58:40.838143238 +0000 UTC m=+0.406652508 container attach a960f4afcc807ed0eae82e3a5b95ae8017f4a3a91be2d4d728fb8b9f1942fc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T18:58:41.175 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:40 smithi175 podman[73564]: 2024-09-10 18:58:40.939870146 +0000 UTC m=+0.508379404 container died a960f4afcc807ed0eae82e3a5b95ae8017f4a3a91be2d4d728fb8b9f1942fc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T18:58:41.426 INFO:journalctl@ceph.osd.5.smithi175.stdout:Sep 10 18:58:41 smithi175 podman[73607]: 2024-09-10 18:58:41.416356076 +0000 UTC m=+0.470291315 container remove a960f4afcc807ed0eae82e3a5b95ae8017f4a3a91be2d4d728fb8b9f1942fc01 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T18:58:41.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:41.282+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:41.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:41.282+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:41.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:41.282+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:15.617906+0000 front 2024-09-10T18:58:15.617921+0000 (oldest deadline 2024-09-10T18:58:40.917898+0000) 2024-09-10T18:58:41.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:41.203+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:41.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:41.203+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:41.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:41.203+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:41.428 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:41.203+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:41.439 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T18:58:41.477 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:41.477 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T18:58:41.478 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T18:58:41.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.6 2024-09-10T18:58:41.797 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 systemd[1]: Stopping Ceph osd.6 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:42.176 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:41.796+0000 7fa23e0fd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:42.177 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:41.796+0000 7fa23e0fd640 -1 osd.6 763 *** Got signal Terminated *** 2024-09-10T18:58:42.177 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:41 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:41.796+0000 7fa23e0fd640 -1 osd.6 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:42.272+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:42.272+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:42.272+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:15.617906+0000 front 2024-09-10T18:58:15.617921+0000 (oldest deadline 2024-09-10T18:58:40.917898+0000) 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:42.179+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:42.179+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:42.179+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:42.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:42 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:42.179+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:43.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:43.299+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:43.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:43.299+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:43.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:43.299+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:15.617906+0000 front 2024-09-10T18:58:15.617921+0000 (oldest deadline 2024-09-10T18:58:40.917898+0000) 2024-09-10T18:58:43.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:43.210+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:43.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:43.210+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:43.678 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:43.210+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:43.678 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:43 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:43.210+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:43.826 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:43.572Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:43.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:43.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:43.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:43 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:43.573Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:44.279+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:44.279+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:44.279+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:15.617906+0000 front 2024-09-10T18:58:15.617921+0000 (oldest deadline 2024-09-10T18:58:40.917898+0000) 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:44.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:44.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:44.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:44.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:44 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:44.247+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:45.676 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:45.278+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:45.278+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:45.278+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:15.617906+0000 front 2024-09-10T18:58:15.617921+0000 (oldest deadline 2024-09-10T18:58:40.917898+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:45.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:45.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:45.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:45.218+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:45.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:45 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:45.218+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:46.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:46.313+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:58:00.115118+0000 front 2024-09-10T18:58:00.115074+0000 (oldest deadline 2024-09-10T18:58:22.415122+0000) 2024-09-10T18:58:46.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:46.313+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.216343+0000 front 2024-09-10T18:58:05.216358+0000 (oldest deadline 2024-09-10T18:58:28.716331+0000) 2024-09-10T18:58:46.677 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6[51350]: 2024-09-10T18:58:46.313+0000 7fa237f10640 -1 osd.6 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:15.617906+0000 front 2024-09-10T18:58:15.617921+0000 (oldest deadline 2024-09-10T18:58:40.917898+0000) 2024-09-10T18:58:46.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:46.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:46.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:46.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:46.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:46.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:46.678 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:46.247+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:46.678 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:46 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:46.247+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:47.495 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:47 smithi175 podman[73692]: 2024-09-10 18:58:47.223669354 +0000 UTC m=+5.530507357 container died 2f15ed4d435495b7a2bfa4e0043c178e6f58de3ebe3687ad39d017ecc9b6d0fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:58:47.496 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:47.217+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:47.496 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:47.217+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:47.496 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:47.217+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:47.496 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:47.217+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:47.496 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:47 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:47.217+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:47.870 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:47 smithi175 podman[73692]: 2024-09-10 18:58:47.593266638 +0000 UTC m=+5.900104638 container cleanup 2f15ed4d435495b7a2bfa4e0043c178e6f58de3ebe3687ad39d017ecc9b6d0fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:58:47.870 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:47 smithi175 bash[73692]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6 2024-09-10T18:58:48.169 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:47 smithi175 podman[73714]: 2024-09-10 18:58:47.869351925 +0000 UTC m=+0.638485932 container remove 2f15ed4d435495b7a2bfa4e0043c178e6f58de3ebe3687ad39d017ecc9b6d0fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2) 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:48 smithi175 podman[73811]: 2024-09-10 18:58:48.118895094 +0000 UTC m=+0.015789798 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:48 smithi175 podman[73811]: 2024-09-10 18:58:48.245472648 +0000 UTC m=+0.142367390 container create c07473ea3ee90bee697f7242eaed3c4c0b794318782d21960e8a2c5c45c980a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:48 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:48.168+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:48 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:48.168+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:48 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:48.168+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:48 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:48.168+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:48.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:48 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:48.168+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:48.927 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:48 smithi175 podman[73811]: 2024-09-10 18:58:48.455991462 +0000 UTC m=+0.352886160 container init c07473ea3ee90bee697f7242eaed3c4c0b794318782d21960e8a2c5c45c980a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T18:58:48.927 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:48 smithi175 podman[73811]: 2024-09-10 18:58:48.461063433 +0000 UTC m=+0.357958130 container start c07473ea3ee90bee697f7242eaed3c4c0b794318782d21960e8a2c5c45c980a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T18:58:48.927 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:48 smithi175 podman[73811]: 2024-09-10 18:58:48.522607993 +0000 UTC m=+0.419502690 container attach c07473ea3ee90bee697f7242eaed3c4c0b794318782d21960e8a2c5c45c980a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:58:48.927 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:48 smithi175 podman[73811]: 2024-09-10 18:58:48.634548812 +0000 UTC m=+0.531443509 container died c07473ea3ee90bee697f7242eaed3c4c0b794318782d21960e8a2c5c45c980a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2) 2024-09-10T18:58:49.214 INFO:journalctl@ceph.osd.6.smithi175.stdout:Sep 10 18:58:49 smithi175 podman[73855]: 2024-09-10 18:58:49.19273328 +0000 UTC m=+0.552756754 container remove c07473ea3ee90bee697f7242eaed3c4c0b794318782d21960e8a2c5c45c980a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T18:58:49.215 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.151+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:49.215 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.151+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:49.215 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.151+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:49.215 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.151+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:49.215 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.151+0000 7fa8356c0640 -1 osd.7 763 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14532.0:1171 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e763) 2024-09-10T18:58:49.220 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T18:58:49.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:49.260 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T18:58:49.260 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T18:58:49.260 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.7 2024-09-10T18:58:49.599 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 systemd[1]: Stopping Ceph osd.7 for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:49.927 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.596+0000 7fa8378a5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:49.927 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.596+0000 7fa8378a5640 -1 osd.7 763 *** Got signal Terminated *** 2024-09-10T18:58:49.927 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:49 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:49.596+0000 7fa8378a5640 -1 osd.7 763 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:58:50.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:50 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:50.140+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:50.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:50 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:50.140+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:50.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:50 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:50.140+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:50.427 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:50 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:50.140+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:50.576 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:50.152Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:50.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:50.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:50.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:50 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:50.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:51.613 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:51.186+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:51.614 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:51.186+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:51.614 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:51.186+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:51.614 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:51 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:51.186+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:52.588 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:52 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:52.176+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:52.588 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:52 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:52.176+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:52.588 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:52 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:52.176+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:52.588 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:52 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:52.176+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:53.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:53.204+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:53.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:53.204+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:53.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:53.204+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:53.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:53 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:53.204+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:53.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:53.573Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:53.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:53.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:58:53.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:58:53 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:58:53.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:58:54.676 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:54.249+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6803 osd.0 since back 2024-09-10T18:57:59.411766+0000 front 2024-09-10T18:57:59.411731+0000 (oldest deadline 2024-09-10T18:58:21.111655+0000) 2024-09-10T18:58:54.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:54.249+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6807 osd.1 since back 2024-09-10T18:58:05.212859+0000 front 2024-09-10T18:58:05.212767+0000 (oldest deadline 2024-09-10T18:58:30.512747+0000) 2024-09-10T18:58:54.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:54.249+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6811 osd.2 since back 2024-09-10T18:58:13.914064+0000 front 2024-09-10T18:58:13.914072+0000 (oldest deadline 2024-09-10T18:58:37.414014+0000) 2024-09-10T18:58:54.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:54.249+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.131:6815 osd.3 since back 2024-09-10T18:58:19.115061+0000 front 2024-09-10T18:58:19.115030+0000 (oldest deadline 2024-09-10T18:58:45.014991+0000) 2024-09-10T18:58:54.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:54 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7[57691]: 2024-09-10T18:58:54.249+0000 7fa8356c0640 -1 osd.7 763 heartbeat_check: no reply from 172.21.15.175:6802 osd.4 since back 2024-09-10T18:58:28.316629+0000 front 2024-09-10T18:58:28.316660+0000 (oldest deadline 2024-09-10T18:58:54.216702+0000) 2024-09-10T18:58:55.421 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:55 smithi175 podman[73939]: 2024-09-10 18:58:55.152936723 +0000 UTC m=+5.668412973 container died 9eb8fd3af6e53b21330b8701f960bfe14f9297f04339a2feb69d7899a160a1d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T18:58:55.676 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:55 smithi175 podman[73939]: 2024-09-10 18:58:55.519412222 +0000 UTC m=+6.034888462 container cleanup 9eb8fd3af6e53b21330b8701f960bfe14f9297f04339a2feb69d7899a160a1d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T18:58:55.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:55 smithi175 bash[73939]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7 2024-09-10T18:58:56.080 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:55 smithi175 podman[73970]: 2024-09-10 18:58:55.762103448 +0000 UTC m=+0.604812169 container remove 9eb8fd3af6e53b21330b8701f960bfe14f9297f04339a2feb69d7899a160a1d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T18:58:56.376 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:56 smithi175 podman[74064]: 2024-09-10 18:58:55.982167224 +0000 UTC m=+0.015479096 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:58:56.376 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:56 smithi175 podman[74064]: 2024-09-10 18:58:56.087825305 +0000 UTC m=+0.121137222 container create 014f2c73668af0846a3438cd6efc18fea0bb291c2b9feebca6f3285db5dbbb2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:58:56.376 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:56 smithi175 podman[74064]: 2024-09-10 18:58:56.290041926 +0000 UTC m=+0.323353798 container init 014f2c73668af0846a3438cd6efc18fea0bb291c2b9feebca6f3285db5dbbb2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T18:58:56.376 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:56 smithi175 podman[74064]: 2024-09-10 18:58:56.294806057 +0000 UTC m=+0.328117929 container start 014f2c73668af0846a3438cd6efc18fea0bb291c2b9feebca6f3285db5dbbb2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T18:58:56.376 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:56 smithi175 podman[74064]: 2024-09-10 18:58:56.375164788 +0000 UTC m=+0.408476673 container attach 014f2c73668af0846a3438cd6efc18fea0bb291c2b9feebca6f3285db5dbbb2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T18:58:56.677 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:56 smithi175 podman[74064]: 2024-09-10 18:58:56.470373678 +0000 UTC m=+0.503685552 container died 014f2c73668af0846a3438cd6efc18fea0bb291c2b9feebca6f3285db5dbbb2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-10T18:58:57.083 INFO:journalctl@ceph.osd.7.smithi175.stdout:Sep 10 18:58:57 smithi175 podman[74107]: 2024-09-10 18:58:57.062222924 +0000 UTC m=+0.582512383 container remove 014f2c73668af0846a3438cd6efc18fea0bb291c2b9feebca6f3285db5dbbb2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T18:58:57.088 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T18:58:57.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:58:57.127 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T18:58:57.127 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T18:58:57.128 DEBUG:teuthology.orchestra.run.smithi131:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T18:58:57.527 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:58:57 smithi131 systemd[1]: Stopping Ceph rgw.foo.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:58:57.826 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:58:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a[69448]: 2024-09-10T18:58:57.525+0000 7f752d82d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:58:57.826 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:58:57 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a[69448]: 2024-09-10T18:58:57.525+0000 7f7531d357c0 -1 shutting down 2024-09-10T18:59:00.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:00.153Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:59:00.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:00.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:59:00.577 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:00 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:00.154Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:59:03.826 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:03.574Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:59:03.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:03.574Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:59:03.827 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:03 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:03.575Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:59:07.827 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:07 smithi131 bash[151710]: time="2024-09-10T18:59:07Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:59:07.827 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:07 smithi131 podman[151710]: 2024-09-10 18:59:07.551639489 +0000 UTC m=+10.228630474 container died c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T18:59:08.166 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:07 smithi131 podman[151710]: 2024-09-10 18:59:07.981132894 +0000 UTC m=+10.658123884 container cleanup c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T18:59:08.166 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:07 smithi131 bash[151710]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T18:59:08.401 DEBUG:teuthology.orchestra.run.smithi131:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T18:59:08.424 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:08 smithi131 podman[151745]: 2024-09-10 18:59:08.165413253 +0000 UTC m=+0.611909235 container remove c0191076c1b5d6123673a14cba3d86e8bc8307ef5430177ebeb03fc2191f0839 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-rgw-foo-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T18:59:08.424 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:08 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:59:08.424 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:08 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service: Failed with result 'exit-code'. 2024-09-10T18:59:08.424 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:08 smithi131 systemd[1]: Stopped Ceph rgw.foo.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:08.424 INFO:journalctl@ceph.rgw.foo.a.smithi131.stdout:Sep 10 18:59:08 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@rgw.foo.a.service: Consumed 3.613s CPU time. 2024-09-10T18:59:08.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:59:08.438 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T18:59:08.438 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T18:59:08.438 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T18:59:08.725 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 systemd[1]: Stopping Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:59:09.023 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.724Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.725Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.725Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:59:09.024 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a[68447]: ts=2024-09-10T18:59:08.725Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:59:09.025 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:08 smithi175 podman[74194]: 2024-09-10 18:59:08.734688054 +0000 UTC m=+0.102767842 container died 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:09.289 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:09 smithi175 podman[74194]: 2024-09-10 18:59:09.121350446 +0000 UTC m=+0.489430232 container cleanup 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:09.290 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:09 smithi175 bash[74194]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:59:09.523 DEBUG:teuthology.orchestra.run.smithi175:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T18:59:09.544 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:09 smithi175 podman[74212]: 2024-09-10 18:59:09.289228209 +0000 UTC m=+0.556295143 container remove 11fae576b7c26db600b4fdddb7d768e69e139b13c851473126fd408f012a2ad0 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:09.544 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:09 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:59:09.545 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:09 smithi175 systemd[1]: Stopped Ceph prometheus.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:09.545 INFO:journalctl@ceph.prometheus.a.smithi175.stdout:Sep 10 18:59:09 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 4.650s CPU time. 2024-09-10T18:59:09.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:59:09.557 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T18:59:09.558 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm rm-cluster --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:59:10.326 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:10 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:10.154Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:59:10.327 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:10 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:10.155Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi131.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.131:8443: connect: connection refused" 2024-09-10T18:59:10.327 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:10 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:10.155Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi175.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.175:8443: connect: connection refused" 2024-09-10T18:59:11.326 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:11 smithi131 systemd[1]: Stopping Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:59:11.713 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:11 smithi131 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a[78872]: ts=2024-09-10T18:59:11.446Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:59:11.714 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:11 smithi131 podman[152179]: 2024-09-10 18:59:11.487937174 +0000 UTC m=+0.147792128 container died 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:11.996 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:11 smithi131 podman[152179]: 2024-09-10 18:59:11.803097471 +0000 UTC m=+0.462952424 container cleanup 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:11.997 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:11 smithi131 bash[152179]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:59:12.326 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:11 smithi131 podman[152196]: 2024-09-10 18:59:11.996176697 +0000 UTC m=+0.505124581 container remove 871fe8dfbdd6a2986fd78d1c713aefeda720e855a536f25c714c7796db7f18a8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:12.326 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:12 smithi131 podman[152196]: 2024-09-10 18:59:12.071854221 +0000 UTC m=+0.580802106 volume remove 712669c0bf1c661cc410b985302a601fe0a777728a8b2cd8cb4f668891bec6d8 2024-09-10T18:59:12.804 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:12 smithi131 systemd[1]: Stopping Ceph node-exporter.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:59:12.804 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:12 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:59:12.804 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:12 smithi131 systemd[1]: Stopped Ceph alertmanager.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:12.804 INFO:journalctl@ceph.alertmanager.a.smithi131.stdout:Sep 10 18:59:12 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.807s CPU time. 2024-09-10T18:59:13.076 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:12 smithi131 podman[152312]: 2024-09-10 18:59:12.803595462 +0000 UTC m=+0.115773358 container died ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:13.426 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 podman[152312]: 2024-09-10 18:59:13.182898835 +0000 UTC m=+0.495076736 container cleanup ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:13.426 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 bash[152312]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T18:59:13.742 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 podman[152329]: 2024-09-10 18:59:13.425638918 +0000 UTC m=+0.625854054 container remove ceab5aa8058692be6c113cd9662c00bc2b321ee636fa1dd1ed4577df07ebd33e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:59:13.742 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:59:13.742 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T18:59:13.743 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 systemd[1]: Stopped Ceph node-exporter.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:13.743 INFO:journalctl@ceph.node-exporter.a.smithi131.stdout:Sep 10 18:59:13 smithi131 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 3.557s CPU time. 2024-09-10T18:59:14.234 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm rm-cluster --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:59:15.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:15 smithi175 systemd[1]: Stopping Ceph iscsi.iscsi.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:59:16.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:15 smithi175 ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a[63110]: debug Shutdown received 2024-09-10T18:59:26.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:25 smithi175 bash[74582]: time="2024-09-10T18:59:25Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:59:26.088 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:25 smithi175 podman[74582]: 2024-09-10 18:59:25.829498016 +0000 UTC m=+10.121925630 container died 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T18:59:26.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:26 smithi175 podman[74582]: 2024-09-10 18:59:26.186666859 +0000 UTC m=+10.479094474 container cleanup 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T18:59:26.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:26 smithi175 bash[74582]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T18:59:26.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:26 smithi175 podman[74606]: 2024-09-10 18:59:26.455281246 +0000 UTC m=+0.624825264 container remove 9af4a6fa44c2661f41b0760094d358a2e17bf555aad03ec3d78b784bbd3acfa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T18:59:26.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:26 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:59:27.109 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:26 smithi175 podman[74653]: 2024-09-10 18:59:26.712566338 +0000 UTC m=+0.110981039 container died ba9ecb40f6d4b1b4f93162b081adc2712e4d884d5821d3389a7ff4217d950ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:59:27.426 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 podman[74653]: 2024-09-10 18:59:27.208352042 +0000 UTC m=+0.606766733 container cleanup ba9ecb40f6d4b1b4f93162b081adc2712e4d884d5821d3389a7ff4217d950ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:59:27.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 bash[74653]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T18:59:27.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T18:59:27.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62975 (conmon) remains running after unit stopped. 2024-09-10T18:59:27.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 74670 (podman) remains running after unit stopped. 2024-09-10T18:59:27.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 systemd[1]: Stopped Ceph iscsi.iscsi.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:27.427 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.103s CPU time. 2024-09-10T18:59:27.834 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:27 smithi175 systemd[1]: Stopping Ceph grafana.a for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:59:27.834 INFO:journalctl@ceph.iscsi.iscsi.a.smithi175.stdout:Sep 10 18:59:27 smithi175 podman[74670]: 2024-09-10 18:59:27.427155546 +0000 UTC m=+0.715627968 container remove ba9ecb40f6d4b1b4f93162b081adc2712e4d884d5821d3389a7ff4217d950ae6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:59:28.135 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:27 smithi175 podman[74795]: 2024-09-10 18:59:27.83332614 +0000 UTC m=+0.101575288 container died 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, distribution-scope=public, name=ubi8, com.redhat.component=ubi8-container, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, vcs-type=git, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8) 2024-09-10T18:59:28.426 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 podman[74795]: 2024-09-10 18:59:28.208340047 +0000 UTC m=+0.476589191 container cleanup 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , vcs-type=git, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, name=ubi8, version=8.5, distribution-scope=public, description=Ceph Grafana Container, release=239.1651231664, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-10T18:59:28.427 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 bash[74795]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a 2024-09-10T18:59:28.821 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 podman[74814]: 2024-09-10 18:59:28.484588197 +0000 UTC m=+0.653563432 container remove 6cbf2b261d086bc77c9712339f9caa9907b818873699b7677683b7836fa13c24 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-grafana-a, vcs-type=git, maintainer=Paul Cuzner , com.redhat.component=ubi8-container, io.openshift.tags=base rhel8, name=ubi8, architecture=x86_64, release=239.1651231664, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, distribution-scope=public, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc.) 2024-09-10T18:59:28.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:59:28.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T18:59:28.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 systemd[1]: Stopped Ceph grafana.a for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:28.822 INFO:journalctl@ceph.grafana.a.smithi175.stdout:Sep 10 18:59:28 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@grafana.a.service: Consumed 11.265s CPU time. 2024-09-10T18:59:29.109 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:28 smithi175 systemd[1]: Stopping Ceph node-exporter.b for 0846fc60-6fa3-11ef-bcea-c7b262605968... 2024-09-10T18:59:29.373 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 podman[74934]: 2024-09-10 18:59:29.108663571 +0000 UTC m=+0.128680252 container died 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:59:29.677 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 podman[74934]: 2024-09-10 18:59:29.471850926 +0000 UTC m=+0.491867606 container cleanup 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:59:29.677 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 bash[74934]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T18:59:29.997 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 podman[74950]: 2024-09-10 18:59:29.714528064 +0000 UTC m=+0.609362217 container remove 2c98a99c00913d59d0eceb6f30ec0331b7daf321b6994db535f42e36176c5965 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-0846fc60-6fa3-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:59:29.997 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:59:29.997 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T18:59:29.997 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 systemd[1]: Stopped Ceph node-exporter.b for 0846fc60-6fa3-11ef-bcea-c7b262605968. 2024-09-10T18:59:29.997 INFO:journalctl@ceph.node-exporter.b.smithi175.stdout:Sep 10 18:59:29 smithi175 systemd[1]: ceph-0846fc60-6fa3-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 3.552s CPU time. 2024-09-10T18:59:30.498 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:59:30.533 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:59:30.559 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T18:59:30.561 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238/remote/smithi131/crash 2024-09-10T18:59:30.562 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:59:30.812 INFO:teuthology.orchestra.run.smithi131.stderr:tar: /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:59:30.813 INFO:teuthology.orchestra.run.smithi131.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:59:30.815 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238/remote/smithi175/crash 2024-09-10T18:59:30.816 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:59:30.856 INFO:teuthology.orchestra.run.smithi175.stderr:tar: /var/lib/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:59:30.856 INFO:teuthology.orchestra.run.smithi175.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:59:30.858 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T18:59:30.858 DEBUG:teuthology.orchestra.run.smithi131:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-09-10T18:59:30.891 INFO:teuthology.orchestra.run.smithi131.stderr:grep: /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T18:59:30.893 INFO:tasks.cephadm:Compressing logs... 2024-09-10T18:59:30.893 DEBUG:teuthology.orchestra.run.smithi131:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:59:30.936 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:: No such file or directory 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/tcmu-runner.log: 63.7% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T18:59:30.964 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T18:59:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T18:59:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T18:59:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T18:59:30.965 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:59:30.966 INFO:teuthology.orchestra.run.smithi131.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T18:59:30.966 INFO:teuthology.orchestra.run.smithi131.stderr:: No such file or directory 2024-09-10T18:59:30.966 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T18:59:30.966 INFO:teuthology.orchestra.run.smithi131.stderr:gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:59:30.966 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T18:59:30.967 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T18:59:30.967 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:59:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T18:59:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T18:59:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T18:59:30.969 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T18:59:30.973 INFO:teuthology.orchestra.run.smithi175.stderr: 91.8% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T18:59:30.975 INFO:teuthology.orchestra.run.smithi175.stderr: 90.9% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:59:30.976 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:59:30.977 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.3.log: 90.8% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:59:31.004 INFO:teuthology.orchestra.run.smithi131.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:59:31.006 INFO:teuthology.orchestra.run.smithi131.stderr: 93.3% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T18:59:31.358 INFO:teuthology.orchestra.run.smithi131.stderr: 90.2% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T18:59:32.110 INFO:teuthology.orchestra.run.smithi175.stderr: 92.4% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T18:59:32.269 INFO:teuthology.orchestra.run.smithi131.stderr: 92.3% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T18:59:33.122 INFO:teuthology.orchestra.run.smithi131.stderr: 91.6% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T18:59:35.953 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T18:59:36.547 INFO:teuthology.orchestra.run.smithi175.stderr: 94.0% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T18:59:36.608 INFO:teuthology.orchestra.run.smithi131.stderr: 93.8% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T18:59:36.768 INFO:teuthology.orchestra.run.smithi131.stderr: 94.1% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T18:59:36.950 INFO:teuthology.orchestra.run.smithi175.stderr: 94.0% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T18:59:37.134 INFO:teuthology.orchestra.run.smithi131.stderr: 93.9% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T18:59:37.305 INFO:teuthology.orchestra.run.smithi175.stderr: 93.9% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T18:59:37.307 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-10T18:59:37.307 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m6.356s 2024-09-10T18:59:37.307 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m23.192s 2024-09-10T18:59:37.307 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.865s 2024-09-10T18:59:37.448 INFO:teuthology.orchestra.run.smithi131.stderr: 94.1% -- replaced with /var/log/ceph/0846fc60-6fa3-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T18:59:37.450 INFO:teuthology.orchestra.run.smithi131.stderr: 2024-09-10T18:59:37.450 INFO:teuthology.orchestra.run.smithi131.stderr:real 0m6.499s 2024-09-10T18:59:37.450 INFO:teuthology.orchestra.run.smithi131.stderr:user 0m27.009s 2024-09-10T18:59:37.450 INFO:teuthology.orchestra.run.smithi131.stderr:sys 0m0.965s 2024-09-10T18:59:37.451 INFO:tasks.cephadm:Archiving logs... 2024-09-10T18:59:37.451 DEBUG:teuthology.misc:Transferring archived files from smithi131:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238/remote/smithi131/log 2024-09-10T18:59:37.452 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:59:41.089 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238/remote/smithi175/log 2024-09-10T18:59:41.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:59:44.383 INFO:tasks.cephadm:Removing cluster... 2024-09-10T18:59:44.383 DEBUG:teuthology.orchestra.run.smithi131:> sudo cephadm rm-cluster --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 --force 2024-09-10T18:59:44.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo cephadm rm-cluster --fsid 0846fc60-6fa3-11ef-bcea-c7b262605968 --force 2024-09-10T18:59:45.393 INFO:tasks.cephadm:Teardown complete 2024-09-10T18:59:45.394 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T18:59:45.403 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T18:59:45.403 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:59:45.407 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T18:59:45.474 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T18:59:45.482 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T18:59:45.483 DEBUG:teuthology.orchestra.run.smithi131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:59:45.485 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:59:45.498 INFO:teuthology.orchestra.run.smithi131.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:59:45.499 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:59:45.502 INFO:teuthology.orchestra.run.smithi131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:59:45.502 INFO:teuthology.orchestra.run.smithi131.stdout:=============================================================================== 2024-09-10T18:59:45.502 INFO:teuthology.orchestra.run.smithi131.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +75us[ +14us] +/- 49ms 2024-09-10T18:59:45.502 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 +1190us[+1190us] +/- 79ms 2024-09-10T18:59:45.502 INFO:teuthology.orchestra.run.smithi131.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -1053us[-1053us] +/- 62ms 2024-09-10T18:59:45.503 INFO:teuthology.orchestra.run.smithi131.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:59:45.504 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:59:45.504 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-10T18:59:45.504 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 24 +66us[ +26us] +/- 49ms 2024-09-10T18:59:45.504 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +1187us[+1147us] +/- 79ms 2024-09-10T18:59:45.504 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -1066us[-1066us] +/- 62ms 2024-09-10T18:59:45.504 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:59:45.505 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T18:59:45.514 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T18:59:45.514 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T18:59:45.522 DEBUG:teuthology.orchestra.run.smithi131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:59:45.622 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:59:45.695 DEBUG:teuthology.orchestra.run.smithi131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:59:45.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:59:45.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:45.753 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T18:59:45.761 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T18:59:45.769 INFO:teuthology.task.internal:Duration was 1999.391274 seconds 2024-09-10T18:59:45.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T18:59:45.778 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T18:59:45.778 DEBUG:teuthology.orchestra.run.smithi131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:59:45.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:59:45.816 INFO:teuthology.orchestra.run.smithi131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:59:45.831 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:59:46.116 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T18:59:46.116 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi131.front.sepia.ceph.com 2024-09-10T18:59:46.116 DEBUG:teuthology.orchestra.run.smithi131:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:59:46.179 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-09-10T18:59:46.179 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:59:46.208 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T18:59:46.208 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:59:46.221 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:59:46.274 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T18:59:46.274 DEBUG:teuthology.orchestra.run.smithi131:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:59:46.289 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:59:46.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T18:59:46.718 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T18:59:46.718 DEBUG:teuthology.orchestra.run.smithi131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:59:46.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:59:46.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T18:59:46.778 DEBUG:teuthology.orchestra.run.smithi131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:59:46.785 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:59:46.806 INFO:teuthology.orchestra.run.smithi131.stdout:kernel.core_pattern = core 2024-09-10T18:59:46.832 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-09-10T18:59:46.885 DEBUG:teuthology.orchestra.run.smithi131:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:59:46.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.900 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:59:46.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:59:46.939 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T18:59:46.949 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T18:59:46.950 DEBUG:teuthology.misc:Transferring archived files from smithi131:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238/remote/smithi131 2024-09-10T18:59:46.950 DEBUG:teuthology.orchestra.run.smithi131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:59:46.991 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900238/remote/smithi175 2024-09-10T18:59:46.992 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:59:47.028 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T18:59:47.029 DEBUG:teuthology.orchestra.run.smithi131:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:59:47.031 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:59:47.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T18:59:47.088 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T18:59:47.088 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T18:59:47.098 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T18:59:47.099 DEBUG:teuthology.orchestra.run.smithi131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:59:47.101 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:59:47.113 INFO:teuthology.orchestra.run.smithi131.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:59 /home/ubuntu/cephtest 2024-09-10T18:59:47.133 INFO:teuthology.orchestra.run.smithi175.stdout: 265707 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:59 /home/ubuntu/cephtest 2024-09-10T18:59:47.134 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T18:59:47.144 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T18:59:47.165 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 1999.3912737369537 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T18:59:47.165 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:59:47.248 INFO:teuthology.run:pass